Batch assembly jobs

,
1

Hello,

I am having a similar issue with same file format and similar size (1.5GB each) but to run Spades for de novo assemby of genome. The job was queued on Wednesday (Jan 3rd 2024) among other 60 jobs. The last one of the batch finished Friday (Jan 5th) and this is the last one, waiting to run solo for almost 48 hours (today is Jan 7th). Looking for answers in previous posts, I usually see that it may take some time and we should wait, but also that a 24 hours wait is acceptable, but something longer than this could be an issue.
This is not the first time that I send a number of jobs to run and the last one, of a larger file, get stuck for quite some time. However, it usually goes after a day or so.
Is there anything I could do besides waiting?
I’m running the jobs in usegalaxy.eu

Best regards
Eduardo

2

Hi @Eduardo_Shimbori

The clusters were probably just busy. Some of your jobs run, some of other people’s jobs run, more of yours, repeat.

Larger jobs will require larger cluster resources, and those tend to the “busiest”. Why? These are already running larger jobs, so take longer to clear out for new jobs. Smaller jobs run on smaller cluster resources, and those turn over jobs quicker. This seems to match your observations.

I don’t know of anything special going on at the EU server except for the holiday. And … holidays tend to be a popular time for people to do data analysis! Any related bottlenecks should clear up soon. The best advice is to use workflows to get everything queued up, and to use the notification functions.

1 Like