Upload data directly into a collection! Solves: No compatible list of pairs dataset collections available

1

I have been using spades for a long time in an older version of galaxy. Now checking out the latest version I am unable to start a job with the standard input files I have. It complains that there are No compatible list of pairs dataset collections available… While there are recognised fastqsanger.gz files which should be compatible…

What am I doing wrong?

I tried to upload a screenshot but the forum did not allow me to either.

It seems in the EU server at least that version (Galaxy Version 3.15.5+galaxy3) works! I think this “collections?” seems to bug things?

2

Welcome @turbine

I’ll try to explain the tool from and how to get the dataset selected!

You have paired end data, correct? Two files, one for the forward reads and one for the reverse reads?

The tool is scanning for a collection shape in your active history.

If you have multiple tabs open in your browser, you might just need to reload your browser to make sure that the active history is correct, so start there!

Then, the tool is looking for a particular shape for the data and that the input data has an accepted datatype format. This makes sure that the parameters on the tool form are correct for your particular data.

How to make a collection → Why collections?

Example for selecting a paired collection for SPAdes

spades-paired-end-input-1
spades-paired-end-input-11186×626 38.9 KB

There are two types of fastq data shapes for the paired end input:

  • interlaced reads: both forward and reverse reads in the same fastq dataset
    • single or multiple datasets with interlaced fastq reads
    • a list collection with one interlaced fastq dataset per sample
  • separate reads: one fastq dataset for forward reads, one for the reverse reads
    • a paired list collection organized by sample, with each containing two datasets

There are several types of fastq data formats:

  • these are the same for any of the selections
  • getting the datatype format is especially import for fastq data since this is metadata that is describing how the quality scores are scaled!
  • loading data with all default settings in the Upload tool is one way to ensure that this is correct! → Getting Data into Galaxy
  • You can also click on the pencil icon for a dataset and redetect the datatype. → FAQ: Detecting the datatype (file format)
  • Tools like FastQC and Fastq info can also help if you need to investigate the format more

This screenshot shows a paired end list collection in the history, and the settings on the tool form that will select it.

spades-paired-end-input-2
spades-paired-end-input-21730×1492 449 KB

This is that history shared if you want to see how it works.

I’m guessing that you have two files – one for the forward and one for the reverse – but these are not in a collection yet? We have lots of help!

3

Thanks for the information! Maybe I am just a very simple user but this makes it a whole lot more complicated at the moment. Maybe I just need to get into it at the moment but in the past it was upload 2 illumina files. Go to the tool select the forward and reverse, click execute.

I am searching for the collection buttons but this is not very intuitive? I actually love the first guy, 1 dataset, one tiny thing and go! I understand if you have 100+ datasets this might become cumbersome but ok. I’ll see if I can figure it out and update a ton of teaching materials :frowning:

4

I figured it out thanks for the video but would it not be great if there was at least some form of backwards compatibility? Now a lot of things break when upping a version. For now ill see if I can downgrade the system and update my material in the coming year…

5

Oh sure, we keep everything that was once available for reproducibility reasons! Very important! We don’t want to break anyone’s workflow! Either an actual workflow or just how you are using the tools.

The prior versions of the SPAdes tool can be navigated with the Versions menu at the top of the tool form. If you were using individual datasets for a pair before, one of these was likely what you were using.

The most recent with this type of input was 3.15.5+galaxy3. → FAQ: Changing the tool version

spades-tool-version-individual-paired
spades-tool-version-individual-paired1450×792 59.9 KB

And, I’ll just add that you can Upload data directly into a collection, too. This is built into the Upload tool now and routes directly to the collection builder. Most is automatic – just name the collection on the next screen. Please ask if you have a question about it! Creating a collection in the history is just another route but most people will load data this way at the start.

upload-into-a-collection
upload-into-a-collection1464×954 67.3 KB

Collections are how to process a lot of data but being able to jump in for exploratory work is important too! We do want feedback about this .. so if you run into any problems or have suggestions, more comments here are great. Me and others helping are also core project developers so you are speaking to the right people! :rocket:

1 Like
6

Then I must have done something wrong as I tried to install an older version of spades it ran into python dependency “hell” for distutils. It’s just unfortunate that support in the last spades version was dropped for individual files I think that caused all the confusion in the first place hah! Thanks for all the feedback :slight_smile:.

7

Ok I think I got it now, you need to create a list of dataset pairs as the section implies “Single-end or paired-end short-reads”. @jennaj where can I put in a request to expand that list for individual paired files and a paired dataset but not a list? As When having 1 dataset it does not always make sense to turn this into a paired dataset or even a list of a datasets of a length of 1… Also I was unable to find a way to unpair a dataset except delete and undelete it again?

8

:slight_smile: i @turbine

For just a few datasets, toggle into Hidden tab in the history. Click on the eye icon to hide or unhide each dataset, or you can use the multi-select to hide/unhide in a batch.

More ways to transform the shape is covered in here. → Hands-on: Using dataset collections / Using dataset collections / Using Galaxy and Managing your Data

Tool panel → Collection Operations is where these tools are. Each has a little graphic to show what is happening but please ask if there is anything else and I’ll try to help. :slight_smile: