Typical run time for jobs? Walltime?


#1

Greetings,

As a new Galaxy user, I’m not sure what typical run times are for jobs. Right now I have a job that has been running for over 24 hours, which consists of a small subset of my samples for parameter testing (Stacks de_novo_map, with 12 samples). Would this run time be considered normal for the job? And is there any “walltime” for usegalaxy.eu that might pose an issue when I run the full analysis on 170 samples?

Many thanks!


#2

Hi @krhaynes, checking the default run times for that tool, it normally finishes within ~2 hours

$ gxadmin tsvquery tool-metrics toolshed.g2.bx.psu.edu/repos/iuc/stacks_denovomap/stacks_denovomap/1.46.0 runtime_seconds | awk '{print $0 / 60 / 60} '| histogram.py
# NumSamples = 69; Min = 0.00; Max = 11.15
# Mean = 1.835991; Variance = 12.667041; SD = 3.559079; Median 0.009444
# each ∎ represents a count of 1
    0.0017 -     1.1169 [    54]: ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
    1.1169 -     2.2321 [     1]: ∎
    2.2321 -     3.3473 [     0]:
    3.3473 -     4.4626 [     0]:
    4.4626 -     5.5778 [     0]:
    5.5778 -     6.6930 [     1]: ∎
    6.6930 -     7.8082 [     2]: ∎∎
    7.8082 -     8.9235 [     2]: ∎∎
    8.9235 -    10.0387 [     8]: ∎∎∎∎∎∎∎∎
   10.0387 -    11.1539 [     1]: ∎

That’s a graph of runtime in hours.

I’m not sure what happened for you. What sort of data are you providing to it? The walltime limit on EU is extremely long (~30 days) so this is unlikely to be an issue :slight_smile:


#3

Hi @hxr,

Two hours is about what I was expecting, so I was surprised by the length of the run.

I have provided gzipped fastq files of forward reads obtained from process_radtags. The 12 files vary from 48.2 MB to 199.7 MB, with an average of 118 MB (~1.4 GB total).

I’m glad to know that walltime won’t be an issue, but am now wondering what is the issue.


#4

Would it be ok if I looked into your account, and at the jobs specifically? It could be the tools do not have enough CPU cores allocated or parameter issues.


#5

Please go ahead–that would be much appreciated!


#6

Hi @hxr, any luck determining what the issue was? The jobs were still running yesterday but this morning it appears they stopped due to an error. I’ll add in a screenshot of the error report below.

Thanks!


#7

Greetings all, just checking again to see if you had any idea what went wrong here–why this ran for ~48 hours instead of the ~2 expected, why some samples show no coverage, and why the catalog construction failed. Any suggestions would be greatly appreciated! :slight_smile:

Thanks!