local galaxy - how to unzip files

mwert · November 11, 2019, 3:59pm

Hello everyone,

So we have installed a local galaxy and most tools we use work perfectly fine. However, a tool I use for subsampling creates a gzipped file as output in the local galaxy. In the online galaxy a unzipped file is created.

Is there a possibility to unzip files on the local galaxy as the other tools in the workflow cannot deal with a zipped file? The online galaxy however seems to not have any issues.

Does anybody know how to resolve this issue?

Thank you very much for your time!

marten · November 11, 2019, 4:10pm

can you link the tool please?

Generally Galaxy supports compressed files, and if a tool requires uncompressed input Galaxy will decompress it on the fly and then supply to the tool. There is a bit of configuration involved so you can decide what formats you support compressed and which should always be decompressed.

fastqsanger will stay compressed in gz/bz2 unless you specifically set the datatype to an uncompressed one when creating/uploading. See auto_compressed_types config flag:

github.com

galaxyproject/galaxy/blob/dev/lib/galaxy/config/sample/datatypes_conf.xml.sample#L122

    
      
          <datatype extension="bowtie_color_index" type="galaxy.datatypes.ngsindex:BowtieColorIndex" mimetype="text/html" display_in_upload="false"/>
          <datatype extension="bowtie_base_index" type="galaxy.datatypes.ngsindex:BowtieBaseIndex" mimetype="text/html" display_in_upload="false"/>
          <datatype extension="csfasta" type="galaxy.datatypes.sequence:csFasta" display_in_upload="true"/>
          <datatype extension="data" type="galaxy.datatypes.data:Data" mimetype="application/octet-stream" max_optional_metadata_filesize="1048576"/>
          <datatype extension="binary" type="galaxy.datatypes.binary:Binary" mimetype="application/octet-stream" max_optional_metadata_filesize="1048576"/>
          <datatype extension="d3_hierarchy" type="galaxy.datatypes.text:Json" mimetype="application/json" subclass="true" display_in_upload="true"/>
          <datatype extension="imgt.json" type="galaxy.datatypes.text:ImgtJson" mimetype="application/json" display_in_upload="True"/>
          <datatype extension="geojson" type="galaxy.datatypes.text:GeoJson" mimetype="application/json" display_in_upload="True"/>
          <datatype extension="data_manager_json" type="galaxy.datatypes.text:Json" mimetype="application/json" subclass="true" display_in_upload="false"/>
          <datatype extension="dbn" type="galaxy.datatypes.sequence:DotBracket" display_in_upload="true" description="Dot-Bracket format is a text-based format for storing both an RNA sequence and its corresponding 2D structure." description_url="https://wiki.galaxyproject.org/Learn/Datatypes#Dbn"/>
          <datatype extension="fai" type="galaxy.datatypes.tabular:Tabular" display_in_upload="true" subclass="true" description="A Fasta Index File is a text file consisting of lines each with five TAB-delimited columns : Name, Length, offset, linebases, Linewidth" description_url="http://www.htslib.org/doc/faidx.html"/>
          <datatype extension="fasta" auto_compressed_types="gz" type="galaxy.datatypes.sequence:Fasta" display_in_upload="true" description="A sequence in FASTA format consists of a single-line description, followed by lines of sequence data. The first character of the description line is a greater-than ('&gt;') symbol in the first column. All lines should be shorter than 80 characters." description_url="https://wiki.galaxyproject.org/Learn/Datatypes#Fasta">
            <converter file="fasta_to_tabular_converter.xml" target_datatype="tabular"/>
            <converter file="fasta_to_bowtie_base_index_converter.xml" target_datatype="bowtie_base_index"/>
            <converter file="fasta_to_bowtie_color_index_converter.xml" target_datatype="bowtie_color_index"/>
            <converter file="fasta_to_2bit.xml" target_datatype="twobit"/>
            <converter file="fasta_to_len.xml" target_datatype="len"/>
            <converter file="fasta_to_fai.xml" target_datatype="fai"/>
            <display file="igv/genome_fasta.xml" inherit="true"/>
          </datatype>
          <datatype extension="fastg" type="galaxy.datatypes.sequence:Fastg" display_in_upload="true" description="fastg format faithfully represents genome assemblies in the face of allelic polymorphism and assembly uncertainty" description_url="http://fastg.sourceforge.net/FASTG_Spec_v1.00.pdf"/>

mwert · November 11, 2019, 4:16pm

Thank you for your reply.

The tool is called: seqtk_sample. The issue is that it just creates the zipped files on the local galaxy but not on the online galaxy. If I use the same tool on the online galaxy, the file is not compressed.

marten · November 11, 2019, 5:30pm

As I said, this is configurable and means that your local configuration (or Galaxy version) differs. What Galaxy version are you using?

marten · November 11, 2019, 7:55pm

this tool support .gz compression (not bz2), and the format of the output is the same as format of the input. So example.fastqsanger.gz will result in output.fastqsanger.gz.