Job delays? How to check your job queues and how to work at multiple public UseGalaxy servers

Dear all, Are you experiencing delays with the upload and runs in Galaxy. org? I have sent a simple fasta file with 7 seqs to upload and its queued “forever”. i also had problems the past couple of days with analyses that never started…

UPDATE: Sharing my history with a friend he was able to use my files and upload the fasta of interest properly and run the analyses. It seems that there is a problem with my account - no idea what could be… please advise.

Thanks in advance,

Maria

Do you maybe have some long-running, unfinished jobs in some history of yours?

Galaxy has a per-user concurrent jobs limit in order to give every user a fair share of the job queue.

If you are reaching this limit, any new job (including uploads) can only start when one of the running jobs finishes.

Dear Wolfgang,

Thanks for your reply. Nope, nothing is running and I have recently cleared my account so my use is very limited (17%). The fasta files that i mentioned have still not been uploaded… No error , not nothin.. It seems just stuck..

I have never experienced this before so I have no idea how to proceed. Until last week everything was working just fine.

Thank you for your time and any possible workarounds you may suggest.
Looking forward to any suggestions.
Best wishes,
Maria

Hi @Maria_Kamilari

Do you want to share the history where the files are uploading? One of us can take a look at the pending jobs. And I can take a look at the entire account directly if you want to share your username (NOT your email address or password – just your public username – which you can make as anonymous as you want under User → Preferences).

Thanks and hopefully we can help to figure out what is going on! :slight_smile:

Hi @Maria_Kamilari

I found your account.

In the default active panel you have two jobs – one is an Upload job of a locally browsed file the other is a tool that retrieves data from NCBI by accession numbers. Both are queued.

BUT, if the hidden panel is toggled on, to show the datasets included in collections, it looks like you have about ~15 queued tblastn jobs. More were recently completed (tblastx, other blast variations).

Most of those jobs run on the largest clusters, and those have busy queues on the public clusters. You probably know this already but using workflows can smooth this queuing process out. Please ask if you would like some help to get started with that from where you are now.

So – overall – I think everything looks fine. You’ll simply need to wait at this server for things to process. BUT, I do have a recommendation. Hop on over to a different server, create an account, and start prepping for your next steps. The core tools and reference data between the UseGalaxy servers are all roughly the same, accounts are distinct with different job/data quotas, and moving data around between them is a few clicks. Very common operations – and avoids “competing against yourself” when working in a different account at the same server doing the exact same work – plus, avoid running into admin problems. In short, different public servers host different public resources, you are welcome to use all, and using all is a good strategy for people with significant computational needs who still want to try at the public sites to see if it will be enough.

More details if interested: ORG has 1 TB scratch, EU has 2 TB → What should I do if my data exceeds the given 250GB of storage? - #2 by jennaj see the transfer link in there or here directly → FAQ: Transfer entire histories from one Galaxy server to another. Exporting to file works fastest for smaller histories since it really is the full dataset but large works too and it is all by URL. Copy the parts of the history you want into a new history, and leave any deleted/purged behind. Then, once you are done, this is the same way you will be downloading your work into your own cloud storage archive locations.

Hope this helps and please ask questions about anything not clear! :slight_smile:

1 Like

Dear Jennifer,

Many thanks for your reply! I have deleted the tblastn jobs and the files uploaded right away :slight_smile: Thanks for the info on changing between servers, I will definitely do this as well in the future.
Meanwhile, I have not utilized the workflows till now. Sounds that this way the jobs can be streamlined. If you could provide some information on this to get started, it would be great.

Thanks again for all your efforts.
Best wishes,
Maria :slight_smile:

1 Like

Hey, that’s great @Maria_Kamilari !!

Ok, for workflows …

  1. Run through this tutorial. I know it seems tedious at the start, that’s is on purpose! But push through. It should take about 30 minutes and at the end you’ll know how to extract, edit and run a workflow. And learn how that workflow gets rid of all the initial tedious stuff! → Hands-on: Galaxy Basics for everyone / Galaxy Basics for everyone / Introduction to Galaxy Analyses

  2. From there, you can check to see if anyone has created a workflow that is close to what you want to do already.

  3. We have some QA at this forum that cover some prior corner case discussions. Later on you can also ask if you run into something new or even just get stuck. workflow

Hope this helps! :rocket: