Galaxy 19.01 drmaa problem switching from galaxy.ini to galaxy.yml

1

Hi,

I upgraded our Galaxy instance from 16.xy to 19.01.
I succeeded to configure it with our deprecated galaxy.ini file.
But I face problems to migrate configuration to galaxy.yml file and set it up to use our SGE 8.1.8 cluster :

  • with galaxy.ini, galaxy jobs are correctly shifted and run by SGE
  • but with galaxy.yml, galaxy jobs keep waiting to be shifted but are not received by SGE.

I believe I must have misconfigured something but can’t find what.
Would you have any hint ?

Here are extracts from :

  • our galaxy.yml :
uwsgi:

  http: :7090
  buffer-size: 16384
  processes: 2
  threads: 4
  offload-threads: 1
  static-map: /static/style=static/style/blue
  static-map: /static=static
  static-map: /favicon.ico=static/favicon.ico
  master: true
  pythonpath: lib
  module: galaxy.webapps.galaxy.buildapp:uwsgi_app()
  thunder-lock: true
  die-on-term: true
  hook-master-start: unix_signal:2 gracefully_kill_them_all
  hook-master-start: unix_signal:15 gracefully_kill_them_all
  py-call-osafterfork: true
  enable-threads: true

galaxy:

  mule: lib/galaxy/main.py
  farm: job-handlers:1
  • our job_conf.xml :
<job_conf>
    <plugins workers="4">
        <plugin id="sge" type="runner" load="galaxy.jobs.runners.drmaa:DRMAAJobRunner">
            <param id="drmaa_library_path">/SGE/8.1.8/lib/lx-amd64/libdrmaa.so</param>
        </plugin>
    </plugins>
    <destinations default="sge_default">
        <destination id="sge_default" runner="sge">
            <param id="nativeSpecification">-V -j n -q short.q</param>
        </destination>
    </destinations>
  • our galaxy.log :
galaxy.jobs.command_factory INFO 2019-04-03 16:42:44,377 [p:14860,w:2,m:0] [DRMAARunner.work_thread-3] Built script [/gs7k1/home/galaxydev/galaxy_19.01/database/jobs_directory/018/18532/tool_script.sh] for tool command [python '/gs7k1/home/galaxydev/galaxy_19.01/tools/data_source/upload.py' /gs7k1/home/galaxydev/galaxy_19.01 /gs7k1/home/galaxydev/galaxy_19.01/database/jobs_directory/018/18532/registry.xml /gs7k1/home/galaxydev/galaxy_19.01/database/jobs_directory/018/18532/upload_params.json 46441:/gs7k1/home/galaxydev/galaxy_19.01/database/jobs_directory/018/18532/dataset_46441_files:/gs7k1/home/galaxydev/galaxy_19.01/database/files/046/dataset_46441.dat]
galaxy.jobs.runners DEBUG 2019-04-03 16:42:44,449 [p:14860,w:2,m:0] [DRMAARunner.work_thread-3] (18532) command is: rm -rf working; mkdir -p working; cd working; /gs7k1/home/galaxydev/galaxy_19.01/database/jobs_directory/018/18532/tool_script.sh; return_code=$?; cd '/gs7k1/home/galaxydev/galaxy_19.01/database/jobs_directory/018/18532';
galaxy.jobs.runners.drmaa DEBUG 2019-04-03 16:42:44,483 [p:14860,w:2,m:0] [DRMAARunner.work_thread-3] (18532) submitting file /gs7k1/home/galaxydev/galaxy_19.01/database/jobs_directory/018/18532/galaxy_18532.sh
galaxy.jobs.runners.drmaa DEBUG 2019-04-03 16:42:44,483 [p:14860,w:2,m:0] [DRMAARunner.work_thread-3] (18532) native specification is: -V -j n -q short.q

Regards.

2 Likes
2

Should be under uwsgi section in the config per Scaling and Load Balancing — Galaxy Project 23.1.1.dev0 documentation

I am not sure that is the root cause but definitely firs thing to try.

1 Like
3

Thank you for the hint.

It was in the wrong section. Works fine now.

1 Like