MultiQC Dataset selector stuck

Uncategorized
1

Hello,

I am experiencing an issue with the dataset selector in Galaxy Europe while configuring MultiQC.

I have approximately 130 STAR-related datasets in my history. When I open the STAR log output selector in MultiQC, the selector initially displays 50 datasets and then shows:

“Loading more… (50 of 130)”

However, it remains at this stage and does not load the remaining datasets, even after waiting.

This makes it difficult to select some of the STAR log outputs that are located later in the history.

I have tested this in different web browsers, but the same problem occurs in all of them.

The issue occurs with:

Tool: MultiQC
Galaxy tool version: 1.35+galaxy4
Galaxy server: usegalaxy.eu

I have also tried reopening the selector and reopening the tool , but the behavior persists.

Is this a known issue with the dataset selector when a history contains many datasets, or is there another recommended way to access the datasets beyond the first 50 entries?

I have attached a screenshot showing the selector stuck at “Loading more… (50 of 130)”.

Additional information

I would also like to add that this issue caused a significant practical problem during the analysis.

Because the dataset selector remained stuck at “Loading more… (50 of 130)”, I had to select the STAR log outputs manually, one by one, while carefully checking each dataset. This took approximately 30 minutes.

Even after doing this carefully, MultiQC processed only part of the intended input in one of the attempts. When datasets were selected individually, I also encountered difficulties ensuring that all samples had been included correctly.

In contrast, when I used the corresponding dataset collections, MultiQC processed the samples correctly and very quickly.

Therefore, the main problem appears to be the dataset selector not loading the remaining entries beyond the first 50. If the selector could simply load all 130 available datasets, the workflow would be straightforward and this problem would essentially disappear.

This is especially important when working with many RNA-seq samples, because manual selection of dozens of individual outputs is both time-consuming and prone to accidental omissions.

Thank you again for looking into this.

image
image1903×530 66.1 KB

2

Hi @aaak

Yes, this is a known limitation is related to the web browser behavior. We are still working on a solution. I’m glad you asked since I don’t think it has been discussed here yet and I’m sure it is frustrating for people!

We agree about all of the clicking being tedious. Using collection folders is the “Galaxy way” to avoid this. It sounds like you already know how to use collections, so I’m going to use this opportunity to explain for others reading who happen to find this topic!

What to do

  1. Put all of your files of the same type into a List collection folder.

  2. Choose the collection input instead.

This is a little FAQ and video that shows how to do create a collection quickly!

When working with so many samples, using name tags will likely be helpful to track samples through tools.

Backing up, if you start off with your reads inside a collection, you can run all of the downstream tools on multiple samples all at the same time. Or, you can add data into a collection later. This is another example video quickly showing how processing with collections works.

More about using collection folders in Galaxy.

Please give this a try and let us know if it works or not! Follow up questions are welcome. :slight_smile:

3

Thank you very much for the detailed explanation and for linking the GitHub issue.

I would like to clarify what I observed.

I am already using dataset collections in this workflow. The difficulty is specifically with the data selector inside the repeated tool input fields.

When I selected individual STAR log datasets in separate repeated STAR input fields, MultiQC was able to run, but because those Galaxy datasets had identical output names, MultiQC treated them as the same sample and the final report contained only one sample row.

When I used the corresponding STAR log collections, MultiQC handled the samples correctly and generated a single combined report with the individual samples shown separately.

However, I was only able to include 21 of the 24 samples, because the selector remained stuck at “Loading more… (50 of 130)” and did not load the additional collection I needed to select.

So the main issue is not the use of collections itself, but the current data-selection/pagination limitation in the tool form, which prevents access to inputs beyond the initially loaded entries.

This also explains why the process became unnecessarily time-consuming: I spent around 30 minutes carefully selecting inputs, but still could not include all 24 samples because the required collection never became available in the selector.

image
image1694×876 96.5 KB

image
image882×1161 49.4 KB

4

Hi @aaak

Thanks for clarifying. I can confirm that the open issue is also about pagination within the multi-select functions on tool forms. The workaround for now (for this specifically) is to use collections instead.

Then for this issue about the sample naming and how MultiQC interprets input data:

When the data is organized into a collection, the element identifier is passed as the sample label name to tools, including MultiQC.

However, using individual files loses this information, and the data will collapse! If your data involves pairs, then there is some special handling to consider, too.

From here, I’m not clear on how these data were created (the internal sample names within the files) and how these are currently organized and input to MultiQC. Would you like to share these details? I can help you to get the samples named uniquely to avoid the somewhat greedly sample merging behavior.