local galaxy - how to unzip files

mwert · November 11, 2019, 3:59pm

Hello everyone,

So we have installed a local galaxy and most tools we use work perfectly fine. However, a tool I use for subsampling creates a gzipped file as output in the local galaxy. In the online galaxy a unzipped file is created.

Is there a possibility to unzip files on the local galaxy as the other tools in the workflow cannot deal with a zipped file? The online galaxy however seems to not have any issues.

Does anybody know how to resolve this issue?

Thank you very much for your time!

marten · November 11, 2019, 4:10pm

can you link the tool please?

Generally Galaxy supports compressed files, and if a tool requires uncompressed input Galaxy will decompress it on the fly and then supply to the tool. There is a bit of configuration involved so you can decide what formats you support compressed and which should always be decompressed.

fastqsanger will stay compressed in gz/bz2 unless you specifically set the datatype to an uncompressed one when creating/uploading. See auto_compressed_types config flag:

github.com

galaxyproject/galaxy/blob/dev/lib/galaxy/config/sample/datatypes_conf.xml.sample#L122


      
            <display file="intermine/intermine_simple.xml"/>
          </datatype>
          <datatype extension="customtrack" type="galaxy.datatypes.interval:CustomTrack"/>
          <datatype extension="bowtie_color_index" type="galaxy.datatypes.ngsindex:BowtieColorIndex" mimetype="text/html" display_in_upload="false"/>
          <datatype extension="bowtie_base_index" type="galaxy.datatypes.ngsindex:BowtieBaseIndex" mimetype="text/html" display_in_upload="false"/>
          <datatype extension="csfasta" type="galaxy.datatypes.sequence:csFasta" display_in_upload="true"/>
          <datatype extension="data" type="galaxy.datatypes.data:Data" mimetype="application/octet-stream" max_optional_metadata_filesize="1048576"/>
          <datatype extension="gz" type="galaxy.datatypes.binary:Binary" subclass="true"/>
          <datatype extension="binary" type="galaxy.datatypes.binary:Binary" mimetype="application/octet-stream" max_optional_metadata_filesize="1048576"/>
          <datatype extension="d3_hierarchy" type="galaxy.datatypes.text:Json" mimetype="application/json" subclass="true" display_in_upload="true"/>
          <datatype extension="imgt.json" type="galaxy.datatypes.text:ImgtJson" mimetype="application/json" display_in_upload="True"/>
          <datatype extension="geojson" type="galaxy.datatypes.text:GeoJson" mimetype="application/json" display_in_upload="True"/>
          <datatype extension="data_manager_json" type="galaxy.datatypes.text:Json" mimetype="application/json" subclass="true" display_in_upload="false"/>
          <datatype extension="dbn" type="galaxy.datatypes.sequence:DotBracket" display_in_upload="true" description="Dot-Bracket format is a text-based format for storing both an RNA sequence and its corresponding 2D structure." description_url="https://wiki.galaxyproject.org/Learn/Datatypes#Dbn"/>
          <datatype extension="fai" type="galaxy.datatypes.tabular:Tabular" display_in_upload="true" subclass="true" description="A Fasta Index File is a text file consisting of lines each with five TAB-delimited columns : Name, Length, offset, linebases, Linewidth" description_url="http://www.htslib.org/doc/faidx.html"/>
          <datatype extension="fasta" auto_compressed_types="gz" type="galaxy.datatypes.sequence:Fasta" display_in_upload="true" description="A sequence in FASTA format consists of a single-line description, followed by lines of sequence data. The first character of the description line is a greater-than ('&gt;') symbol in the first column. All lines should be shorter than 80 characters." description_url="https://wiki.galaxyproject.org/Learn/Datatypes#Fasta">
            <infer_from suffix="fa" />
            <converter file="fasta_to_tabular_converter.xml" target_datatype="tabular"/>
            <converter file="fasta_to_bowtie_base_index_converter.xml" target_datatype="bowtie_base_index"/>
            <converter file="fasta_to_bowtie_color_index_converter.xml" target_datatype="bowtie_color_index"/>
            <converter file="fasta_to_2bit.xml" target_datatype="twobit"/>

mwert · November 11, 2019, 4:16pm

Thank you for your reply.

The tool is called: seqtk_sample. The issue is that it just creates the zipped files on the local galaxy but not on the online galaxy. If I use the same tool on the online galaxy, the file is not compressed.

marten · November 11, 2019, 5:30pm

As I said, this is configurable and means that your local configuration (or Galaxy version) differs. What Galaxy version are you using?

marten · November 11, 2019, 7:55pm

this tool support .gz compression (not bz2), and the format of the output is the same as format of the input. So example.fastqsanger.gz will result in output.fastqsanger.gz.

Topic		Replies	Views
producing gzip compressed datafiles usegalaxy.eu support metadata , datatype	3	1379	May 7, 2020
Cannot open zipped file -- Solution: load individual datasets, not multi-file archives upload	1	607	February 24, 2022
Creating tool which requires specific input file extension server-admin , tool-dev	2	366	September 14, 2022
New datatype in galaxy but default file extension when download galaxy-local	0	246	December 28, 2020
Paste Link to Upload Data -- Uploading tar archives by URL usegalaxy.org support upload	6	1181	February 2, 2024

local galaxy - how to unzip files

Related Topics