Hello everyone,
I wonder why, since few days, running jobs have a very long waiting time. In detail: the genome assembly step in my custom-made pipeline is in a “grey-colored” status for many hours. Would it be associated to busy servers in this period? Thanks.
1 Like
Hello, I highlighted the same issue days before, also in uploading files. I guess is a sarver issue. It then got fixed over the weekend, and I realized over the night there are no problems in running jobs… so I guess it’s getting busy and the server is slowing down.
Assembly tools are computationally expensive, so tend to run longer than simple tools. That means the clusters running other people’s jobs need to clear before running your jobs. The public resources are seriously significant but not infinite!
Some of your jobs run, some of other people’s jobs, more of your jobs, repeat.
Waiting for a day or so for access to a large cluster node is not unusual. Those take more time to turn over current jobs and free up for new work. The best strategy when using public resources is to: start up your workflow’s jobs, set a notification, then come back once everything is done.
Now, you are welcome to try clicking on tools one by one, but that means the queue timing will seem even more extended due to the pauses in between human-started jobs. Getting everything queued at once with a workflow would definitely improve what is happening if you are running work manually now.
The holidays seem to be very popular time for extra data analysis! So, things are usually a bit busier in Nov/Dec every year, then tend to clear up in January as more people get back to doing more of what they normally do 
. But the advice above applies year round: get jobs queued as quickly as possible when using the public computational resources. And if you ever have really time sensitive work, consider the private options with no/less resource competition (apply your grant funding, etc).
Bonus: To see how many jobs are running at the UseGalaxy.eu server, go to the homepage and find the links to the graphics. These are real time! We’ll be adding those for other public servers … but all look about the same. Thousands of jobs a minute!