Extremely Long Queue Times RNA-seq Data Processing

Hi @kfelsen1

We received your email message at this server’s support mailing list and are reviewing your specific account/jobs. Please do not delete any queued jobs, or they enter the job queue at the back again, extending wait time. We will send you more feedback about your specific work via email (help about potential input issues, etc).

In general, the Galaxy Main https://usegalaxy.org server is under very heavy usage. Due to heavy load and related factors, some queued jobs are delayed longer than usual. Our administrator is working to move those forward. Delays are expected for all work at most public Galaxy servers, including this one. There are many existing topics at this forum that explain the job execution delays across tools, analysis goals, and public resources. The basic underlying reason for delays is that there are simply more people using public resources, including Galaxy servers, for learning and computational work.

Large, computationally intensive work with time-sensitive deadlines are usually not appropriate for public Galaxy servers. Public Galaxy servers are shared resources. The good news is that there are many ways to use Galaxy!

A cloud version of Galaxy is often a good solution – the GVL version of Galaxy using AWS is a popular choice – simplified web-based administration, on-demand resource allocation, and the like. AWS has always offered grants for research work and that program was expanded, last time I checked, in an effort to help the many more people that are learning, teaching, and focusing on computational projects online. Galaxy itself is always free – but commercial storage/computational resources are not.

Full details:

Regarding:

Please note that the “quota space” represents the amount of data storage available. It is unrelated to the resources required to execute jobs/analysis. All tools hosted at usegalaxy.org already have the maximum computational resources allocated. If jobs fail for exceeding resources (red dataset with a memory or walltime aka “execution time” error), that means there is an input problem or the work really is too large to run at the public resource.

How to check: Troubleshooting resources for errors or unexpected results

@eduardofox2 Longer queue times are expected, especially for compute-intensive tools like BWA. If you have paused jobs, check the upstream jobs that are inputs. Were these completed successfully? You may need to examine data nested inside of dataset collections (some elements may have run successfully, and some not). Those upstream jobs can be rerun. And if you are using dataset collections, those reruns can replace the original failed results – tool forms include an option for this, located right above the job submission button. That said, if you would also like a closer review, send me a direct message here and we can troubleshoot more from there.

Thanks!

2 Likes