Server Issue? Jobs running slow -- Public server delays plus Using Galaxy choices

usegalaxy.org support
1

Is there an issue with the servers? It’s taking me a whole day just to upload some files, another day to run HISAT2, and I’ve been waiting since yesterday afternoon for my Stringtie data.

1 Like
2

Hi @Jodi_D

Uploading data will take longer than usual if you are now working from a home internet connection instead of an institution/commercial connection. Home internet services are almost always optimized for data download speeds, not upload speeds.

Try running a “speedtest” to find out what your connection speeds are. Contact your ISP provider to find out what options are available to increase “up” speeds. There are several “speedtest” options – an internet search will find those appropriate for where you are located, or you can ask your ISP provider for guidance.

The internet, in general, is a bit slower for most right now, or at least not what it was before. More people working over home wifi plus phone connections, more people streaming entertainment during the day, etc.

The Galaxy Main https://usegalaxy.org server is under very heavy load. This means that jobs will stay in the queued stage (grey dataset) longer. The execution stage (yellow/peach dataset) is about the same as before for most tools. There are simply more people using the server, for a variety of reasons, including distance learning.

The best strategy is to queue up jobs, then allow them to complete. Avoid deleting and restarting jobs in hopes of job running faster. Restarting a job places it at the back of the queue again, extending wait time. Only delete and rerun jobs if you already know there is some problem with the inputs/settings originally used. If you haven’t used Workflows before, this might be a good time to learn/start using them.

FAQ for jobs/datasets: Datasets and how jobs execute

Many GTN Tutorials include workflows in the tutorial itself and most have a workflow that runs through all steps (except for data loading) that can be used as examples/templates. Those under the topic “Introduction” cover simple cases for the “how-to”. Other topics cover intermediate/advanced methods for customization.

Note: There are a few tools running with reduced memory resources. These are listed in the top banner on the server. The banner will update or be removed as the situation changes. Reduced memory doesn’t impact how long a job takes to queue or execute but it can result in larger jobs failing for resources (even if they ran Ok before).

If work is urgent or you just want jobs to execute run faster, setting up your own Galaxy server is an option. Public servers are not the only choice. Cloudman is popular for scientists as it requires very little administration, resources can be scaled up for your own needs, and all of that (except for the initial step) is GUI based with built-in help. Galaxy itself is always free but commercial storage and computational resources are not. AWS has always offered grants for learning/research work, and they have extended that (or did last time I checked). I added a few tags to your post about Cloudman – please review for more details if this interests you, or see the links below for all choices.

Delays are frustrating, but I hope this helps to explain the factors involved in the different kinds of delays most public Galaxy servers are experiencing right now (not just usegalaxy.org), plus what you may choose to do to mitigate delays.

Good question! Hopefully, the Q&A helps others to understand the current circumstances.

Best, Jen

1 Like
3

Thanks, Jen. I really appreciate your response

1 Like
4

Did you ever get your StringTie to work? I’ve completed trimming and mapping; everything has gone relatively quickly, now I’m using StringTie for assembly- its been over 10 hours and it’s still processing. I’m just wondering if this is normal or if I should suspect something with my inputs is wrong.

1 Like
5

Sorry @nicole_g

Missed your post earlier.

The servers are still very busy. Which public Galaxy server is “fastest” can vary.

I suspect your jobs already completed by now – but people are still asking if there are delays in new posts and are being redirected to prior Q&A through the tag “server-side-delays”.

Usegalaxy.org is a bit slower right now – it depends on the tool – and that will change over time.

  • If a dataset (job) is gray, then the inputs have not been evaluated yet. Meaning, delays are not due to problems with the inputs.

  • If a dataset (job) is blue, then that means it attempted to run but there was a problem with the inputs, and you’ll need to review the jobs that are creating the inputs – something went wrong upstream. The expanded dataset will have more comments about what went wrong but you’ll need to do the full detective work to find and fix the input problems.

  • If a dataset (job) is red, that means it errored out. That could indicate an input problem. Try at least one rerun to eliminate a transient server issue, then if it fails again, check your inputs.

Help for checking inputs: