Help installing DRMAA Python for use with Slurm?

usegalaxy.org support
,
1

Hello,

I am trying to install Galaxy so that it sends jobs to slurm. I have used the tool ParallelCluster to create my cluster, which has slurm installed in /opt/slurm. I have also downloaded slurm-drmaa, as the docs suggest. I installed it with the following commands: ./configure && make, sudo make install.

I downloaded Galaxy by using git clone on my main directory.
I then changed my Galaxy config files to interface with slurm. My current galaxy/config/job_conf.yml file is set up like this:

<job_conf>
<plugins workers="10">
    <plugin id="slurm" type="runner" load="galaxy.jobs.runners.slurm:SlurmJobRunner">
        <param id="drmaa_library_path">/usr/local/lib/libdrmaa.so</param>
    </plugin>
</plugins>
<handlers default="handlers">
    <handler id="main" tags="handlers">
        <plugin id="slurm"/>
    </handler>
</handlers>
<destinations default="slurm_part">
    <destination id="slurm_part" runner="slurm">
</destination>
    <destination id="slurm_part_12" runner="slurm">
        <param id="nativeSpecification">--ntasks=12</param>
    </destination>
</destinations>
<tools>
    <tool id="flexbar" destination="slurm_part_12"/>
    <tool id="tophat2" destination="slurm_part_12"/>
    <tool id="bowtie2" destination="slurm_part_12"/>
    <tool id="bwa_wrapper" destination="slurm_part_12"/>
</tools>
</job_conf>

Now, when I run sh run.sh, I get this error:

galaxy.jobs INFO 2019-07-22 20:34:02,874 [p:29606,w:0,m:0] [MainThread] Handler 'main' will load specified runner plugins: slurm
galaxy.jobs.runners DEBUG 2019-07-22 20:34:02,874 [p:29606,w:0,m:0] [MainThread] Loading SlurmRunner with params: {'drmaa_library_path': '/usr/local/lib/libdrmaa.so'}
galaxy.jobs.runners.state_handler_factory DEBUG 2019-07-22 20:34:02,875 [p:29606,w:0,m:0] [MainThread] Loaded 'failure' state handler from module galaxy.jobs.runners.state_han               dlers.resubmit
galaxy.jobs.runners.drmaa INFO 2019-07-22 20:34:02,875 [p:29606,w:0,m:0] [MainThread] Overriding DRMAA_LIBRARY_PATH due to runner plugin parameter: /usr/local/lib/libdrmaa.so
    
Traceback (most recent call last):
      File "lib/galaxy/webapps/galaxy/buildapp.py", line 49, in app_factory
        app = galaxy.app.UniverseApplication(global_conf=global_conf, **kwargs)
      File "lib/galaxy/app.py", line 204, in __init__
        self.job_manager = manager.JobManager(self)
      File "lib/galaxy/jobs/manager.py", line 26, in __init__
        self.job_handler = handler.JobHandler(app)
      File "lib/galaxy/jobs/handler.py", line 51, in __init__
        self.dispatcher = DefaultJobDispatcher(app)
      File "lib/galaxy/jobs/handler.py", line 945, in __init__
        self.job_runners = self.app.job_config.get_job_runner_plugins(self.app.config.server_name)
      File "lib/galaxy/jobs/__init__.py", line 599, in get_job_runner_plugins
        rval[id] = runner_class(self.app, runner['workers'], **runner.get('kwds', {}))
      File "lib/galaxy/jobs/runners/drmaa.py", line 68, in __init__
        (exc.__class__.__name__, str(exc)))
    ImportError: The Python drmaa package is required to use this feature, please install it or correct the following error:
    ImportError: No module named drmaa

I followed the instructions on the DRMAA Python github page, and installed the package using pip install drmaa, but I don’t know how to configure it. Can anyone help?

2

Did you ever find a solution for this?

3

I was able to solve this by activating the .venv and just pip installing drmaa in case anyone else runs across this.

2 Likes