Maker and Braker 3 running for half a week - and still going

usegalaxy.eu support
,
1

Good day!

Last week, around Thursday, I launched genome annotation with BRAKER3 and then with Maker on the day after. It’s been half a week - but the process is still going… Seems like it shouldn’t be that long. I also noticed that I have unlimited storage for some reason

Here’s a link to my history:

Thanks for the help in advance!

2 Likes
2

Welcome @drbrownse

The jobs appear to be running! The EU server will allow you to process really massive jobs. These will run until the tool dies or completes. Now, some jobs can exceed practical processing limits (wouldn’t process anywhere, not just through Galaxy) and will be killed but that is rare.

The best advice I have is to allow this to process. If the job is killed and restarted, the new job just begins again, and the original job logs will be lost.

Since I’m not personally sure about processing estimates for the soybean genome, and was interested in a bit of context, and I asked an AI what it thought. See below and expand if interested, and of course the citations could be hallucinated! But a search across other direct resources leads me to think it is mostly correct. Complex genomes take longer to process. This maker job had sequences input, not alignments, and did not have longer reads included.

It has only been 4-5 days. I’d let these process to see what happens! :scientist:

Why your account has unlimited storage is curious! If you requested extra quota space, the administrators may have granted this for short term work. You could write back to them and ask (this forum is public, account issues are better handled via email). The email contact is at the bottom of the server homepage: contact at usegalaxy dot eu. Ping @wm75 can you help?

AI query against Grok (example)

prompt: maker genome annotation job soybean genome estimated runtime

A MAKER genome annotation job for the soybean genome, using standard parameters, is expected to take several days, potentially weeks, to complete. The exact time depends on the size of the genome, the computational resources available, and the specific configuration of the MAKER pipeline. [1, 2, 3]

Factors Affecting Runtime: [3, 3]

  • Genome Size: Plant genomes, including soybean, are relatively large compared to other eukaryotes, meaning the time to annotate them can be measured in months rather than hours. [3, 3]
  • Computational Resources: MAKER can be parallelized and runs faster on more powerful machines with more cores. [2, 2, 4, 4, 5]
  • Data Inputs: Using pre-aligned evidence (e.g., protein and mRNA alignments) can significantly reduce runtime, as MAKER does not have to perform these alignments itself. [6, 6]
  • MAKER Configuration: The specific parameters used in the MAKER pipeline, including the choice of gene prediction algorithms, can impact runtime. [2, 2, 7, 7]
  • Training Data: If MAKER is trained on existing data, it may run faster and with better accuracy on subsequent runs. [1, 1, 8, 8]

Example: [4, 4]

A MAKER run on a smaller genome (129MB) using 8 nodes took 7 hours. However, for a larger plant genome like soybean, the time can be significantly longer. In a more detailed example, a study using MAKER on S. mediterranea ran on a single-core MAC laptop at a rate of 4.1 h/Mb. This suggests that a larger genome would take proportionally longer to annotate on a single core. [3, 3, 4, 4, 9, 9]

In summary, while MAKER is designed to be efficient, annotating a large plant genome like soybean can still take several days to weeks, depending on the specific circumstances and computational resources available. [1, 2, 3]

Generative AI is experimental.

[1] Yandell Lab - Software - MAKER

[2] In-depth description of running MAKER for genome annotation. · GitHub

[3] https://academic.oup.com/plphys/article/164/2/513/6112783

[4] https://bioinformaticsworkbook.org/dataAnalysis/GenomeAnnotation/Intro_To_Maker.html

[5] MAKER2: an annotation pipeline and genome-database management tool for second-generation genome projects - PMC

[6] https://www.yandell-lab.org/publications/pdf/maker_current_protocols.pdf

[7] Genome Annotation and Curation Using MAKER and MAKER-P - PMC

[8] MAKER: an easy-to-use annotation pipeline designed for emerging model organism genomes - PubMed

[9] MAKER: An easy-to-use annotation pipeline designed for emerging model organism genomes - PMC

1 Like
3

Thank you for your answer!

It still seems a bit weird to me that it takes so long as one of my BRACKER3 tasks has finished. I also know that I supplied my earliest BRACKER3 task with wrong RNA data (forgot to check XS in RNA STAR) and I expected it to fail (it didn’t but it hasn’t finished either). I wonder if the data I used for Maker could have some mistakes as well. But if it’s alright I’ll simply wait some more.

Regarding the storage limit - I haven’t requested an increase, but it seemingly happened after I launced Maker for the first time. It first increased to 2TB (from 250GB), then it jumped to unlimited. I’m not complaining and I don’t really plan to use more than 200-250 GB, but it’s still just weird :man_shrugging:

4

@drbrownse there was an issue with one of your BRACKER3 jobs and we restarted this one for you now. The remaining jobs seem to be genuinely running still.

As for your observation of unlimited quota on your account:
thank you very much for reporting this. We had some major configuration changes on Galaxy Europe last week and it seems quota config got accidentally removed during the process.
Should work now again as intended and we really appreciate your honest report about this issue :100:

3 Likes
5

Thanks! Alright, I’ll wait

As for the storage - always glad to help back) Galaxy servers are amazing! :fire:

2 Likes