Typical run time for jobs? Walltime?

usegalaxy.eu support
#1

Greetings,

As a new Galaxy user, I’m not sure what typical run times are for jobs. Right now I have a job that has been running for over 24 hours, which consists of a small subset of my samples for parameter testing (Stacks de_novo_map, with 12 samples). Would this run time be considered normal for the job? And is there any “walltime” for usegalaxy.eu that might pose an issue when I run the full analysis on 170 samples?

Many thanks!

1 Like
#2

Hi @krhaynes, checking the default run times for that tool, it normally finishes within ~2 hours

$ gxadmin tsvquery tool-metrics toolshed.g2.bx.psu.edu/repos/iuc/stacks_denovomap/stacks_denovomap/1.46.0 runtime_seconds | awk '{print $0 / 60 / 60} '| histogram.py
# NumSamples = 69; Min = 0.00; Max = 11.15
# Mean = 1.835991; Variance = 12.667041; SD = 3.559079; Median 0.009444
# each ∎ represents a count of 1
    0.0017 -     1.1169 [    54]: ∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
    1.1169 -     2.2321 [     1]: ∎
    2.2321 -     3.3473 [     0]:
    3.3473 -     4.4626 [     0]:
    4.4626 -     5.5778 [     0]:
    5.5778 -     6.6930 [     1]: ∎
    6.6930 -     7.8082 [     2]: ∎∎
    7.8082 -     8.9235 [     2]: ∎∎
    8.9235 -    10.0387 [     8]: ∎∎∎∎∎∎∎∎
   10.0387 -    11.1539 [     1]: ∎

That’s a graph of runtime in hours.

I’m not sure what happened for you. What sort of data are you providing to it? The walltime limit on EU is extremely long (~30 days) so this is unlikely to be an issue :slight_smile:

1 Like
#3

Hi @hxr,

Two hours is about what I was expecting, so I was surprised by the length of the run.

I have provided gzipped fastq files of forward reads obtained from process_radtags. The 12 files vary from 48.2 MB to 199.7 MB, with an average of 118 MB (~1.4 GB total).

I’m glad to know that walltime won’t be an issue, but am now wondering what is the issue.

1 Like
#4

Would it be ok if I looked into your account, and at the jobs specifically? It could be the tools do not have enough CPU cores allocated or parameter issues.

1 Like
#5

Please go ahead–that would be much appreciated!

1 Like
#6

Hi @hxr, any luck determining what the issue was? The jobs were still running yesterday but this morning it appears they stopped due to an error. I’ll add in a screenshot of the error report below.

36%20AM
Screen Shot 2019-01-31 at 9.47.36 AM.png818×738 145 KB

Thanks!

#7

Greetings all, just checking again to see if you had any idea what went wrong here–why this ran for ~48 hours instead of the ~2 expected, why some samples show no coverage, and why the catalog construction failed. Any suggestions would be greatly appreciated! :slight_smile:

Thanks!

#8

Hi,

since the initial message of this thread by @krhaynes was quite generic (basically “how long do things take on galaxy.eu”), I thought I would share my recent experience as a new user so that people get an idea for the generic tools I have been using:

  • “Gene body coverage” (RSeQC package): 39-64 hours for 6 BAM files.

  • “Transcript Integrity Number” (RSeQC package): 25-41 hours for 6 BAM files.

  • Mapping of human paired-end RNA-seq data using hisat2: 3-4 hours / sample.

  • “MarkDuplicates” (Picard package): 25-50 min / BAM file.

  • “Read Distribution” (RSeQC package): 15-20 mins / BAM file.

  • “BamCoverage” (deepTools package): 15-20 mins to turn a BAM file into a bigwig file.

  • “Convert GTF to BED12”: 10 min / file.

Bye.