put() and get() in jupyterlab notebooks in galaxy

[hxr]

There are lots of tutorials covering the usage of ITs, but none covering building ITs.

get and put are some special magic in the jupyter notebook we build. We configure a special environment for the notebook which loads those commands: docker-jupyter-notebook/ipython-profile.py at a48b767cd7d030b743605bb7984eda5eae76eb69 · bgruening/docker-jupyter-notebook · GitHub

So your tool XML looks fine, but it could be something about the specific container image you’re using. In that container it SHOULD be loaded by default in the default user’s home dir.

I see that you’re missing this statement:

export HOME=/home/jovyan/ &&

there’s a strong change that could be causing it. Generally looking at your XML it has diverged quite far from the in-galaxy jupyter notebook, you might try diffing those files and seeing if some of those changes you’ve made are relevant to the issues.

If it’s ok for everyone, maybe let’s keep the conversation here, it’s a bit easier to see what’s going on than scrolling through hundreds of messages to tease apart the thread.

[maikenp]

Thanks, yes I much prefer disucssing here, the gitter is not ideal for discussions!

Let’s see thanks, I will add the export.

Could you provide a link to the in-galaxy jupyter notebook?

For our notebook docker image, we used as a basis: GitHub - bgruening/docker-jupyter-notebook: Jupyter running in a docker container. This image can be used to integrate Jupyter into Galaxy but added some more software.

[maikenp]

For our notebook docker image, we used as a basis: GitHub - bgruening/docker-jupyter-notebook: Jupyter running in a docker container. This image can be used to integrate Jupyter into Galaxy but added some more software.

The xml copy-pasted in this post is the one from the installation found in /srv/galaxy/server/tools/interactive/interactivetool_jupyter_notebook.xml - with one change - the <url>ipython/lab</url> which had to be changed to <url>lab</url> and some outdated ServerApp replacement to NotebookApp - see some earlier discussion on gitter.

[root@galaxy-hepp interactive]# diff /srv/galaxy/server/tools/interactive/interactivetool_jupyter_notebook.xml.orig interactivetool_jupyter_notebook.xml
1,4c1,5
< <tool id="interactive_tool_jupyter_notebook" tool_type="interactive" name="Interactive Jupyter Notebook" version="0.1">
<     <requirements>
<         <container type="docker">quay.io/bgruening/docker-jupyter-notebook:2021-03-05</container>
<     </requirements>
---
> <tool id="interactive_tool_jupyter_notebook" tool_type="interactive" name="General Jupyter Notebook" version="0.1">
>   <description>Default empty notebook, reuse a previous one, or upload a new</description>
>   <requirements>
>     <container type="docker">quay.io/bgruening/docker-jupyter-notebook:2021-03-05</container>
>   </requirements>
8c9
<             <url>ipython/lab</url>
---
>             <url>lab</url>
35c37
<             jupyter lab --allow-root --no-browser  &&
---
>             jupyter lab --allow-root --no-browser  --ServerApp.token=''  &&
46c48
<                 jupyter lab --allow-root --no-browser --NotebookApp.shutdown_button=True &&
---
>                 jupyter lab --allow-root --no-browser  --ServerApp.token='' &&

[maikenp]

Answering myself: I can find the most recent jupyter_notebook here: galaxy/tools/interactive at dev · galaxyproject/galaxy · GitHub

[maikenp]

Update, using the newest toolwrapper ( ) the put function is now recognized. Still having some issues using it, will get back with details.

[maikenp]

Would you like me to create a new help issue - since now put() is indeed recognized, but I get this:

[maikenp]

The key point being I think the use of the quay.io/bgruening/docker-jupyter-notebook:ie2 image.

[hxr]

Ok! Progress Did you set galaxy_infrastructure_url, this can be important for the commands to connect to the galaxy server.

[maikenp]

Dont think so. Where would I set that, and what should the value be?

[maikenp]

Ah, configuration option, right, lets see…

[maikenp]

So, I set
galaxy_infrastructure_url: http://localhost:8000
since I see in our nginx setup:

[root@galaxy-hepp config]# cat /etc/nginx/conf.d/galaxy-gie-proxy.conf
server {
    # Listen on port 443
    listen       443 ssl;
(.... various ssl settings)
    client_max_body_size 2G;
    proxy_intercept_errors on;

    fastcgi_buffers 16 16k;
    fastcgi_buffer_size 32k;

    #Proxy all requests to the GIE Proxy application
    location / {
       proxy_redirect off;
       proxy_http_version 1.1;
       proxy_set_header Host $host;
       #proxy_set_header X-Real-IP $remote_addr;
       proxy_set_header Upgrade $http_upgrade;
       proxy_set_header Connection "upgrade";
       proxy_pass http://localhost:8000;
    }
}

But in the galaxy.conf I see:

[root@galaxy-hepp config]# cat /etc/nginx/conf.d/galaxy.conf 
server {
  listen 443 ssl;
  server_name  galaxy-hepp.hpc.uio.no;
  
.....(various settings)

  location / {
    uwsgi_pass 127.0.0.1:8080;
    uwsgi_param UWSGI_SCHEME $scheme;
    include uwsgi_params;
  }
}

Which would be the correct to use?

I read from the config doc pages that the default is http://localhost:8080

With the http://localhost:8000 the same error message occurs as the one in the screenshot above.

[maikenp]

@hxr do you have some advice on what would be the expected galaxy_infrastructure_url based on our nginx setup?

In the tool wrapper for the interactive tool there is

entry point port 8888 and galaxy_web_port 8080. Should I be using any of these for galaxy_infrastructure_url?

Thanks!
Maiken

[hxr]

Hi @maikenp sorry for the delay, too many things going on.

Infrastructure url should really be the address you access galaxy at, I see listen 443 in your nginx settings, so for example EU sets it to

galaxy_infrastructure_url: 'https://usegalaxy.eu'

Then the container will use the external address to contact galaxy’s API which is a lot more reliable than trying to figure out the internal networking path to reach galaxy (what it initially tries within the container)

[maikenp]

Thanks a lot for getting back to me, and yes, understand completely the delay.

Ok, I understand, thanks. Will give that a try once we have our system up and going again (going through some reinstallations!)

[maikenp]

Ok, unfortunately that did not help at all. Exact same error now using
galaxy_infrastructure_url: https://galaxy-hepp.hpc.uio.no

What else could be wrong?

[maikenp]

Could there be some

• selinux issues?
• firewall issues?
• docker argument issues?

I have tried looking for hints in various log files, but not really finding anything of interest. So not at all sure where I should be looking.

If it is relevant: the galaxy server sends the job via slurm to a compute node, and jupterlab runs inside the docker container on this compute node.

Also not that (not sure if it is a hint or not) that in the docker container I see:

root@d8087e912320:/storage/galaxy/jobs_directory/001/1251/working/jupyter# curl https://galaxy-hepp.hpc.uio.no/
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

[maikenp]

Hi again, sorry to use the ping, but we would love to get the put() and get() working @hxr

Could the issue be related to some missing docker options? We have this currently in our job_conf.xml

[hxr]

The container has a DEBUG option, I believe, I should have thought of that sooner. If you set that environment variable, then the get/put commands will print out additional logging information that might help you diagnose the issue.

I think you’ll need to set that in the tool xml.

[hxr]

Try adding something like <environment_variable name="DEBUG">True</environment_variable>

[hxr]

xref galaxy_ie_helpers/galaxy_ie_helpers at master · bgruening/galaxy_ie_helpers · GitHubinit.py#L12