Put() not working in jupyter notebook on local server

,
1

I’ve cloned the latest 26.0 release and trying, but failing, to use put() from a Jupyter notebook on my local server. And apologies, as I think I’ve gone through this previously - if so, my notes fail me.

  1. I see there are multiple notebook options - is one preferred? (I tried the *notebook.xml one)

interactivetool_jupyter_notebook_1.0.0.xml interactivetool_jupyter_notebook_1.0.3.xml
interactivetool_jupyter_notebook_1.0.1.xml interactivetool_jupyter_notebook.xml

  1. I’ve attached my config/galaxy.yml and job_conf.yml, fwiw

  2. here’s my attempted use (after creating a “foo.txt”) and error:

Exception                                 Traceback (most recent call last)
<ipython-input-5-63aa956a47a9> in <module>
----> 1 put("foo.txt")

/opt/conda/lib/python3.8/site-packages/galaxy_ie_helpers/__init__.py in put(filenames, file_type, history_id)
    111
    112     history_id = history_id or os.environ['HISTORY_ID']
--> 113     gi = get_galaxy_connection(history_id=history_id)
    114     for filename in filenames:
    115         log.debug('Uploading gx=%s history=%s localpath=%s ft=%s', gi, history_id, filename, file_type)

/opt/conda/lib/python3.8/site-packages/galaxy_ie_helpers/__init__.py in get_galaxy_connection(history_id, obj)
     98     ### Fail ###
     99     msg = "Could not connect to a galaxy instance. Please contact your SysAdmin for help with this error"
--> 100     raise Exception(msg)

Thanks!

2

Hi @rheiland

The v.26.0 release is still in development! :hammer_and_wrench:

The current master is v25.1.1

The most current supported version of the notebook is interactivetool_jupyter_notebook_1.0.3.xml.

Please give this a try with the updated version combination and let us know what happens!

Also – integration testing is happening right now for v.26.0! If you found something that works in the master and fails in the pending release branch, this is important and can be reported right now directly to the /galaxy repository under Issues. If you specify the comparison details at the top of the issue the release team will help to triage, or you can post the link back here and I’ll toggle the flag.

I hope this helps! :slight_smile:

3

Thanks for pointing out 26.0 is in dev. When I download 25.1.1 and attempt to build (sh run.sh), I end up with a rabbit hole error that I had encountered last summer with 25.0 and never resolved. Do you have any more specific tips than the generic yarn url?

[12:42:53] Finished 'default' after 1.21 min
$ GXY_BUILD_SOURCEMAPS=1 NODE_ENV=production webpack
Browserslist: browsers data (caniuse-lite) is 11 months old. Please run:
  npx update-browserslist-db@latest
  Why you should do it regularly: https://github.com/browserslist/update-db#readme
Browserslist: caniuse-lite is outdated. Please run:
  npx update-browserslist-db@latest
  Why you should do it regularly: https://github.com/browserslist/update-db#readme
Killed
error Command failed with exit code 137.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 137.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
ERROR: Galaxy client build failed. See ./client/README.md for more information, including how to get help.
(galaxy-venv) (base) heiland@yoga:~/dev/galaxy-25.1.1$
4

Hi @rheiland

The message about the yarn version can be ignored (probably!). Instead, this is usually about the process memory exceeding the current allocations.

Please see this post for details. → caniuse-lite is outdated error during run.sh build - #9 by pjlombardo

For more choices when starting up, whether with sh run.sh or not, this is the definitive documentation that updates with each release:

From here, there are two primary options

  1. Build the client somewhere that has a lot of memory, then move it to the VM where you want to deploy the service.

  2. Choose a different make client option, or directly set a shell variable for the NODE* memory limit available. The topic above explains where this option was polished a bit more – also here directly.

The steps for option 2 with direct memory set would be something like

  • create the home directly /my-server/galaxy and cd into it
  • export NODE_OPTIONS="--max_old_space_size=4096" to set the limit to 4 GB (or what you have available)
  • sh run.sh

The process can still overflow – setting the swap is probably needed for the full install if the memory is less than ~ 10 GB (I can confirm this minimum for this exact release, and update back here!)

All if this could be added to Get Galaxy | Galaxy Hub since it comes up every so often! TODO item for me!

Please give this a try and let us know what happens! We can troubleshoot more. Knowing more about the environment would help. :woman_scientist:

5

Thanks! Apparently the NODE_OPTIONS did the trick. I’m running now and on to the next issue. I’ve done:

(base) heiland@yoga:~/dev/galaxy-25.1.1/config$ cp galaxy.yml.interactivetools galaxy.yml
(base) heiland@yoga:~/dev/galaxy-25.1.1/config$ cp job_conf.yml.interactivetools job_conf.yml

(base) heiland@yoga:~/dev/galaxy-25.1.1/config$ cp tool_conf.xml.sample tool_conf.xml

--> edit tool_conf.xml to have notebook 1.0.3 (even though not listed in the .sample)
  </section>
  <section id="interactivetools" name="Interactive tools">
    <tool file="interactive/interactivetool_jupyter_notebook_1.0.3.xml" />
  </section>

(base) heiland@yoga:~/dev/galaxy-25.1.1$ source ~/galaxy-venv/bin/activate
(galaxy-venv) (base) heiland@yoga:~/dev/galaxy-25.1.1$



(galaxy-venv) (base) heiland@yoga:~/dev/galaxy-25.1.1$ export NODE_OPTIONS="--max_old_space_size=4096"
(galaxy-venv) (base) heiland@yoga:~/dev/galaxy-25.1.1$ sh run.sh

but when I try to Run the Notebook IT (1.0.3), I get:
Firefox can’t establish a connection to the server at localhost:4002.

6

Hi @rheiland

The tool_conf.xml seems ok. I’m comparing to the configuration at UseGalaxy.org here:

So, this is probably related to the server port.

Default would be this for the server, assuming the tool launched from the same server has the host you just created.

localhost:8080:80

And the remaining parts are configured in galaxy.yml starting here:

This tutorial includes instructions for abstracting this more but the explainations about what each variable means and choices behind them are valid no matter how you do this!

Glad you are getting more done with your IT!