DNA seq paired-end read collection running slow (taking days)

I’m have been trying to map a collection of 96 paired-end fastqsanger files onto a human genome using BWA-MEM. I’ve also tried Bowtie2, and map with minimap2. Reads were trimmed using trimmomatic. I’ve been trying for a couple days to run these jobs and wondering if there is a faster way to do this or if I’m doing anything wrong?

-Z

2 Likes

Hi @zrs1

When using public Galaxy servers (or really, any Galaxy server), it is best to leave queued jobs queued. Each will start up as resources become available. Avoid deleting and restarting, unless you need to modify inputs/settings, or you’ll lose the original placements in the job queue, further extending wait time.

The usegalaxy.org server and many other public servers are very busy right now. Mapping jobs require the use of larger cluster nodes which tend to be the busiest. It will take some time to process all 96 jobs.

More details about how jobs queue and execute are in this recent Q&A: Job Stuck in Queue

Alternatives include setting up your own Galaxy server. These prior Q&A topics have basic details and link-outs to all options. A commercial cloud is not your only choice but does involve less administration/local resources. Commercial servers -- Galaxy Choices && Galaxy Cloudman or GVL?.

To find more Q&A about Galaxy choices, please search this forum with the keyword “choices” or “cloudman”.

Thanks!

Thank you! I let it run for a day. All the jobs paused before completing and won’t restart when I resume them. I tried logging out and back in again. Do you know any way around this issue?

1 Like

Hello,

There were some recent server issues at a few different public Galaxy servers that may have impacted your jobs.

However, a “paused job” (light blue with a link) usually indicates an input problem.

A rerun may be the quickest solution if this was a server-side problem. But check the other items and let us know if you need more help. Where you are using Galaxy will matter for how to best proceed.

Thanks!

Update: Additional server updates/fixes were applied in the last few hours at https://usegalaxy.org.

If your job failed, rerunning is the best/fastest option.

If a job newly started fails from this time point forward, please try a rerun but also let us know here. Please confirm this is the server where you are working in your reply. Including the error message is also helpful.

We sincerely apologize for the recent troubles!