I have a BRAKER3 3.0.8+galaxy2 job that has been running since 14 July 2026. A previous submission of the same analysis remained running for more around a month until I cancelled it. Comparable BRAKER3 analyses for other genomes of the same genus (5 species) completed within approximately one to three days.
I have verified that the FASTA and BAM are valid and mutually consistent. The BAM passes Samtools integrity checks, and standalone StringTie 2.2.1 completed on the entire BAM in 47.5 seconds. The Galaxy live-console API is disabled for this job (not a choice on my end), the metrics endpoint returns no metrics, and the common-problems endpoint reports no empty or duplicate inputs.
I am running the job on default settings, with the genome file, an ordered BAM file, and the Viridiplantae-proteins.fasta provided by BRAKER3 developers. Here are some relevant screenshots for the job.
It is inside GeneMark‑ETP’s ProtHint stage, in the protein‑to‑genome spliced alignment step.
The reason why your job is taking longer than expected is because some unfortunate internal events that also coincide with galaxy downtime on the 27th. Here is a detailed log of condor events (just FYI):
> Initial OOM eviction (14 Jul, 02:20): the job first ran on a slot with only 4096 MB allocated and was killed by the cgroup memory limit within 3 minutes. Galaxy’s automatic retry mechanism released and resubmitted it with progressively larger memory allocations (12 GB, then 36 GB), after which it ran uninterrupted for the next 13 days, with RSS climbing steadily from ~4 GB to ~23 GB — a normal footprint for this stage on a large genome, not a leak.
> Network‑level interruption (27 Jul, 10:25–12:27): the connection between the submit host and execute host dropped (“Local schedd and job shadow died”). Condor attempted to reconnect twice but the lease expired after 2400 s, so the job was evicted (Code 1008) and automatically rescheduled — this is infrastructure‑side, not a BRAKER or data problem.
> Resumed on new host (27 Jul, 12:36). Because braker.pl was invoked with --useexisting, it correctly resumed from the completed checkpoints rather than restarting from scratch — consistent with the process tree already being deep in ProtHint rather than back at GC‑content/AUGUSTUS setup. It has been running continuously on this host since 10 days ago.
Thank you so much for the detailed reply. If it’s not a bother I have one further question.
As it’s taking unexpectedly long, I am considering running BRAKER3 for this species locally, but I have refrained from doing so thus far to keep the program versions for all the species consistent. Is the 3.0.8 version on Github and the BRAKER3 3.0.8+galaxy2 versions functionally the same? It might sound a redundant question, however I didn’t want to assume.
