I’m running the Pre.cluster tool (mothur, Galaxy Version 1.39.5.0) on a
16S mothur pipeline (23 samples, RedSeaSeawater_16S_Mothur pipeline history).
The job has been stuck in “running” status for over 15 hours with no progress
(multiple attempts - job IDs 415-424, dataset 407 and 409 as inputs,
diffs=2, match=1).
I initially had a disk quota issue (97-98% used) which I resolved by
deleting/purging old datasets and relocating some to 30-day storage
(now at 48% quota used). But even after freeing up space and restarting
the job fresh (job 424, started Aug 24 9pm), it’s still stuck running
with no progress.
Could you help me check if this job is actually processing or stuck on
the cluster side?
Job Content API ID: f9cad7b01a472135081291cc03cab713
History API ID: bbd44e69cb8906b53b2db3aff7d0fb6a
Thanks for sharing your history! That was very helpful. I think I can offer a few suggestions here, but please let us know whether they actually help.
The group information will be important when running this tool. Without group information, mothur pre.cluster is processing all ~2.3 million sequences together instead of dividing them into much smaller processing batches (per group).
The training job provides a good example of this: it processes the data by group, with each group containing only hundreds to a few thousand sequences at a time. You can find a link to the relevant tutorials at the bottom of the tool form.
Your dataset may be larger than the training example, but dividing the sequences into group batches will still make each individual clustering task much smaller. The tool’s ability to use additional processor cores also depends on having group information, which can help it process the data faster and reduce the chance of reaching a job time limit.
I also noticed that you changed Number of mismatched bases (diffs) from the Galaxy default of 1 to 2. With such a large dataset, allowing more differences can make the clustering step considerably more computationally demanding.
You can try this at UseGalaxy.eu, where different resources may be available, but the group information will be a factor no matter where the job is run. Combined with the diffs=2 setting and the very large number of sequences, this may be difficult for mothur to process successfully anywhere.
I would first review how the group information should be carried forward into this step and test again using the default parameters.
In short, with the current inputs and settings, this job is likely to continue running until it reaches the maximum job time limit, just as the first job did, and then end with empty results. Different tools handle resource limits differently; this one exits somewhat more gracefully, so the failure is less obvious.
