I’m having issues running Kraken2 and Spades on usegalaxy.eu, I had a spades that I had initiated yesterday, and this morning it hasn’t moved from Gray. Having same issues with Kraken2.
I’ve split my work up between usegalaxy.org and usegalaxy.eu, I’m working on using Kraken2, but since they have different databases utilized I’m having to switch back and forth. I find the 2 newer databases very useful as they fill some voids that the other databases have. The EuPathDB-46 on usegalaxy.org and the Mycobacterium V1 on usegalaxy.eu
The server is processing work but it is very busy. Be sure to leave your work queued or you will lose your place! New jobs are always added back to the end of the queue.
UseGalaxy.org is also very busy! We don’t have the server statistics posted publicly but I just looked and it is about the same.
And, if I am understanding correctly, we need to populate some more indexes! Let me know if I have this correct and I’ll help to make the request. It probably won’t be immediate but we can at least get it started.
if both or either of you would like to post back your Public name at UseGalaxy.org, we can look closer at your accounts to make sure nothing more is going on.
Log into your account and find this under User → Preferences → Manage Information → Public name
Do NOT post your email address here publicly! Your public name is enough, and we never need your password (no actual admin ever should!).
If you can do this quickly we can do this before the weekend!
HI Jennifer, I’m primarily having issues on usegalaxy.eu, I had deleted my Kraken2 run as it sat over 24hours without any movement. The last few days I haven’t been able to run Kraken2 or Spades, but everything else I have been using appears to be working fine. My public name there is jpcnorthwest
Thank you for reaching out and offering your support.
It seems that the server at usegalaxy.eu is currently busy when running Kraken2 and Shovill/Spades, although I’ve been able to run other tools successfully. I’ve noticed these issues since the start of the week. I have a Kraken2 job that has been running for 12 hours and is still ongoing. Should I abort it?
Lastly, I have a quick question as a Galaxy newbie: would long and extended-running or queue times affect the quality or reliability of the output data? My public name is zssgh.
The UseGalaxy.eu is very busy. I see a lot of Kraken2 jobs and related tools running. You both should leave those jobs queued, otherwise they may never get a chance to process.
If I go the EU server homepage, and click into the statistics (how to) I see
From this data right now, in very general terms, I’d say that there is quite a bit of metagenomics work going on right now, and that there is maybe a training (so, smaller jobs that run quickly) along with a lot of workflows running (real data, these usually run longer) plus people using the tools directly.
The server will balance that load and process the work fairly. The rules are a bit complicated and cluster node allocation on the server can be dynamic… but this is “fair” and as balanced as possible, meaning everyone has an equal chance to move up in that queue line.
So, with all the context in mind
Every time a job is deleted and rerun, that new job goes back to the very end of that queue line.
If you delete often enough, those jobs may never have a chance to move up in the queue far enough along to get to the processing stage.
You can run other jobs even if some are queued in your account.
The best advice is to get work queued then let it process. It is the only way to get real work done under this kind of resource competition at a public cluster. Often the same is true in private cluster situations like a university server but those might also have “priority users”. At public servers, all public users have equal priority.
Using collections (folders of similar data) and/or workflows (tools strung together to process the inputs in a sequence) can help. Yes, it keeps the history organized, but most importantly all the jobs submitted are scheduled together – that means they are put into one of these “waiting lines” all at once, right at the start. With the new workflow graphics, you can even go to your workflow reports and watch that happening.
The alternative, clicking tool-by-tool, dataset-by-dataset, will also queue your work but that is click-by-click, and so much more tedious, and your data is not “in line” for the next tool until you manually start that job, when it could have be in line that whole time if a workflow was used at the start. This way of running real data will always take much longer to complete because of all the time gaps between data being “ready” from an upstream job and the next job actually getting into the queue.
Since I can only check accounts at the server where I am an administrator, let’s ask the EU administrators for some more feedback. There does seem to be a bit of a heavy load for these metagenomics tools in particular. Hi @wm75 is this all expected right now? Thanks!
