Deseq2 and deg_annotate header confusion

Hi ,

I ran deseq2 and then used deg_annotate with a GTF to add annotations to the tabular output of deseq2.

  • The deseq2 output in galaxy shows headers
  • When I download the file the headers are gone. Looks like galaxy adds these headers when displaying the data only?
  • The headers are completely gone after deg_annotate.
  • The tutorial asks to add headers manually which is very cumbersome for new user
  • Is there a way the headers can added by deg_annotate and also kept in the downloaded file. The downloaded file is what people usually share use locally.
    Thanks

Hello @microfuge

I ran an updated test to make sure nothing changed recently.

With that context, for your questions. Or, scroll down to the bottom to see my recommendation!

I am not seeing any headers or column numbers in the default view for the tabular files.

This is a generic datatype for “tab separated values”, so column counts 1, 2, .. N are the only “value” and can interfere with data processing if included as a printed line.

tabular format

Then, gtf files have known column values. Strict formatting does not include these (gtf specification).

Maybe up to 80% of the discussions involving Bioconductors tools have some content issue, including “custom headers”.

gtf data format

The Annotate DESeq2/DEXSeq output tables annotation tool’s output does not have any headers. This is just a tabular file format again.

variable-length output columns

tabular format

I agree with this for production use! The workflow can be modified to add in a custom header (hide intermediate outputs).

For training, the GTN tutorials themselves will usually not automate everything as there are little extra manual steps embedded in these for training purposes (to slow students down from just plugging in data + workflow = result without actually reviewing the steps!).

I understand editing a workflow starts to become advanced! Scientists will be usually working without headers during processing, then adding custom headers later if wanted. But you could prepare this for the students?

Yes! You could create a customized workflow for your training that includes steps for adding a header baked in?

:warning: Warning that this can make the custom workflow somewhat “brittle”. A change in the annotation tool’s output columns would need to be coupled with the Add line to file tool’s text content. Maybe put an annotated box in the workflow canvas with a note?

And I would suggest standardizing the format of the header by running the output through the Convert delimiters to TAB tool (make sure all whitespace is reduced to a single tab!).

Something like this to replace the tutorial step at Hands-on: Reference-based RNA-Seq data analysis / Reference-based RNA-Seq data analysis / Transcriptomics (# hands-on-add-column-names)

  1. workflow editor (instructor as part of prep)
  2. annotation output → add header → standardize format
  3. hide intermediate outputs, give the final output a meaningful name, add in some annotation about the dependencies between these steps
  4. test the workflow, then capture the invocation share link. this provides students with the custom answer key history and workflow to reuse?


Summary

  • For students, adding in the header directly avoids introducing workflow editing too soon.
  • For “production” use, this can be avoided entirely with workflow customization but it does introduce a dependency.
  • For Galaxy use, the tool wrapper itself could potentially be updated to add in an optional header based on the advanced settings (use the attributes to label the variable columns) but this would produce a file that will likely need to have the header stripped out to avoid errors, or at least for some use cases.

I know this isn’t exactly what you wanted! Please let me know your thoughts. We can make the request to the tool developers to see what they think. Or you can start an issue up, share it back here, and link in this topic for context in the ticket itself. :hammer_and_wrench:

Thanks so much @jennaj for the detailed and through dissection and resolutions of my questions!

Deseq2 does add headers but they are only visible in the tabulator visualization

It looks like galaxy stores these headers not in the data itself but in some sort of metadata entry. The dataset is even marked as having a header

The solutions you provide work for me. But I find the header situation still cumbersome and error-prone for new users.

  • A dataset should clearly mark that if it has header or not
  • The downloaded file should have these headers or an option to exclude them
  • The default view should show these headers
  • A tabulor editor (like ethercalc) will be a great addition for Galaxy

Thanks Again!