Hello there!
I had an active paired collection including 25 datasets. Then I discarded one of the datasets using filter collection tool. Now, when I want to run cutadapt on the filtered collection (inluding 24 datasets), I get an error for each of the datasets:Input dataset ‘x.fastq.gz’ was deleted before the job started, Input dataset ‘x’ was deleted before the job started
Unfortunately, I can not share my history due to the privacy considerations. I would be grateful if someone could help me.
1 Like
I’d like to review this. I’m an administrator at UseGalaxy.org. Will look for your account directly and then message you directly if I can’t find it. Please leave everything in the current state if at all possible. New data is fine, but don’t delete anything more please so I can review the original states under the hood.
1 Like
Hi. Hop all is well with you.
Did you checked my account?What was wrong with it?
In the case you couldn’t find my account, I will send you the history privately.
1 Like
Hi @mmomeni Please see my direct message to the original account you used to ask the question here. I still need for you to give the name of the history a distinct name.
1 Like
Update
I’ve tested the Filter Collection function and it seems to be working Ok.
My test history is here if you want to compare the steps and how this is done on a very simple example. https://usegalaxy.org/u/jen-galaxyproject/h/test-filter-collection
Some more tips:
- A collection is a folder. It doesn’t contain any data itself and consumes no quota space.
- A dataset is a file. This does contain your data. Purging these is what frees up space.
- The trash can icon (delete function) for a collection has two options:
* Delete just the collection folder (a data structure that references files)
* Delete the collection folder and the datasets inside of it (data structure and the actual files) - Deleted files can be restored (made active) or purged (data entirely removed)
- Delete collection folders from the active tab in the history panel. One step. Once gone, gone forever. But you can re-create it… and the files that were inside will still be in the hidden tab. You can unhide those individually or in batch, then group them into a new collection folder, or do anything else you can do with files.
- Use the gear icon in the history panel to purge specific classes of files in batches. That could be all deleted files, all hidden files, and others.
- Clean up data as you go, especially if you have a lot of files.
- Copy data into new histories for different steps if you think you’ll be curating the content like this. Copies of data in your own account do not consume extra quota space. Copies are technically just references to the original primary file, but if that original is purged, a copy can become the primary file. If you have one copy in any history == you have that data file in your account. Try a search under User → Datasets if it seems lost.
Example…
If the goal is to purge the “discarded” datasets after filtering a collection, consider deleting the target collection (“discarded” folder) and the datasets inside of it using the trash can icon, then use the gear icon to “purge all deleted datasets”. If you just delete the collection folder, those files are still in your account and you’ll need to find them in the hidden tab to do more. Or, you could unhide all datasets if you don’t mind those all showing up in the active tab.
So, lots of flexibility. This might seem hard but is worth learning. Anyone can import my shared history above to play around in.
Much more about Collection Operations can be found in our tutorials here → https://training.galaxyproject.org/training-material/search2?query=collections
Hope that helps!
1 Like
Hi @jennaj
one of our users reported an issue with collections. Files were used in two collections. One collection was deleted (I guess with elements). The files have gone from the second collection, as well.
I can check if the issue still exist.
Kind regards,
Igor
Nope, cannot reproduce the issue anymore. It seems it was fixed.
Igor
1 Like