# Infinitely spawning ghost jobs (JobHandlerQueue.monitor\_thread) **URL:** \ **Category:** Uncategorized\ **Tags:** devops-administration, galaxy-local, troubleshooting, devops-gxadmin\ **Created:** [June 5, 2020, 1:50pm UTC](https://help.galaxyproject.org/t/infinitely-spawning-ghost-jobs-jobhandlerqueue-monitor-thread/3851 "2020-06-05T13:50:14Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![annabeldekker](https://sea2.discourse-cdn.com/flex020/user_avatar/help.galaxyproject.org/annabeldekker/32/1294_2.png) [@annabeldekker](https://help.galaxyproject.org/u/annabeldekker)\ **Post date:** [June 5, 2020, 1:50pm UTC](https://help.galaxyproject.org/t/infinitely-spawning-ghost-jobs-jobhandlerqueue-monitor-thread/3851/1 "2020-06-05T13:50:14Z")
Dear GalaxyHelp, I have a problem with my local instance of galaxy (20.01). I suddenly noticed it was not starting any new jobs or uploading files (staying grey forever). When I looked at the galaxy log, it was constantly starting new jobs, but the database/job\_directory folders galaxy created seem ‘empty’ (empty working and output folder). As this is a development instance, I tried to completely remove the tmp and job\_directory. Still, if I restart galaxy, it will try to start new jobs. This instance doesn’t consist of users that started any jobs, so it is very confusing. I must say that I am working on an automated updating of libraries via the bioblend api. I’m curious if it is possible that this api invokes certain jobs that are stuck in a loop? After some dependency checks which it usually does, my logfile looks like this: Starting server in PID 11717. serving on http://_ **xxx** _:8080 galaxy.model.database\_heartbeat DEBUG 2020-06-05 13:52:26,856 [p:11717,w:1,m:0] [database\_heartbeart\_main.web.1.thread] main.web.1 is config watcher galaxy.jobs.mapper DEBUG 2020-06-05 13:52:27,216 [p:11717,w:1,m:0] [JobHandlerQueue.monitor\_thread] (80530) Mapped job to destination id: local galaxy.jobs.handler DEBUG 2020-06-05 13:52:27,232 [p:11717,w:1,m:0] [JobHandlerQueue.monitor\_thread] (80530) Dispatching to local runner galaxy.jobs DEBUG 2020-06-05 13:52:27,253 [p:11717,w:1,m:0] [JobHandlerQueue.monitor\_thread] (80530) Persisting job destination (destination id: local) galaxy.jobs DEBUG 2020-06-05 13:52:27,253 [p:11717,w:1,m:0] [JobHandlerQueue.monitor\_thread] (80530) Working directory for job is: /home/galaxy/galaxy/database/jobs\_directory/080/80530 galaxy.jobs.runners DEBUG 2020-06-05 13:52:27,278 [p:11717,w:1,m:0] [JobHandlerQueue.monitor\_thread] Job [80530] queued (45.707 ms) galaxy.jobs.handler INFO 2020-06-05 13:52:27,288 [p:11717,w:1,m:0] [JobHandlerQueue.monitor\_thread] (80530) Job dispatched galaxy.jobs.mapper DEBUG 2020-06-05 13:52:27,297 [p:11717,w:1,m:0] [JobHandlerQueue.monitor\_thread] (80531) Mapped job to destination id: local galaxy.jobs.handler DEBUG 2020-06-05 13:52:27,316 [p:11717,w:1,m:0] [JobHandlerQueue.monitor\_thread] (80531) Dispatching to local runner galaxy.jobs DEBUG 2020-06-05 13:52:27,337 [p:11717,w:1,m:0] [JobHandlerQueue.monitor\_thread] (80531) Persisting job destination (destination id: local) galaxy.jobs DEBUG 2020-06-05 13:52:27,339 [p:11717,w:1,m:0] [JobHandlerQueue.monitor\_thread] (80531) Working directory for job is: /home/galaxy/galaxy/database/jobs\_directory/080/8 0531 galaxy.jobs.runners DEBUG 2020-06-05 13:52:27,389 [p:11717,w:1,m:0] [JobHandlerQueue.monitor\_thread] Job [80531] queued (72.643 ms) galaxy.jobs.handler INFO 2020-06-05 13:52:27,408 [p:11717,w:1,m:0] [JobHandlerQueue.monitor\_thread] (80531) Job dispatched galaxy.jobs.mapper DEBUG 2020-06-05 13:52:27,436 [p:11717,w:1,m:0] [JobHandlerQueue.monitor\_thread] (80532) Mapped job to destination id: local galaxy.jobs.handler DEBUG 2020-06-05 13:52:27,469 [p:11717,w:1,m:0] [JobHandlerQueue.monitor\_thread] (80532) Dispatching to local runner etc. creating more and more job directories continuously. Did any of you every experience something similar? I am on the edge of reinstalling the development instance, but I want to avoid this happening again of course. I was wondering if someone has an idea of forcing to stop this process for now. Thank you so much in advance! Annabel Dekker ---
**Author:** ![annabeldekker](https://sea2.discourse-cdn.com/flex020/user_avatar/help.galaxyproject.org/annabeldekker/32/1294_2.png) [@annabeldekker](https://help.galaxyproject.org/u/annabeldekker)\ **Post date:** [March 3, 2021, 8:54am UTC](https://help.galaxyproject.org/t/infinitely-spawning-ghost-jobs-jobhandlerqueue-monitor-thread/3851/2 "2021-03-03T08:54:32Z")
I used gxadmin ([GitHub - galaxyproject/gxadmin: Handy command line utility for Galaxy administrators](https://github.com/galaxyproject/gxadmin)) to purge the database queue of my galaxy instance! ---
**Author:** ![jennaj](https://sea2.discourse-cdn.com/flex020/user_avatar/help.galaxyproject.org/jennaj/32/27_2.png) [@jennaj](https://help.galaxyproject.org/u/jennaj)\ **Post date:** [March 3, 2021, 4:52pm UTC](https://help.galaxyproject.org/t/infinitely-spawning-ghost-jobs-jobhandlerqueue-monitor-thread/3851/3 "2021-03-03T16:52:45Z")
> [@annabeldekker](#): > > gxadmin **gxadmin** to the rescue! The utility is @hexylena’s creation 🏆 🏆 🏆 🏆 🏆 And, thank YOU @annabeldekker for posting the solution back plus including the docs link so others can find out about it. ✅ ✅ ✅ ✅ ✅