Yes, both configuration flavours are currently allowed. You would need to find the correct YAML part. E.g. here: galaxy/lib/galaxy/config/sample/job_conf.sample.yml at dev · galaxyproject/galaxy · GitHub
I’ve tried multiple permutations of the “–add-host” arg in the job_conf.yml, but whatever I try, it seems to prevent my IT from actually starting. It stays stuck on the “Waiting for… to become available”
docker_run_extra_arguments: --add-host localhost:host-gateway --platform linux/amd64
I’m not sure where to go from here. I was really hoping there was an example of an IT, in a Docker container, in a local Galaxy instance, which successfully used the put/get, i.e., could correctly access the env vars in the IT’s .xml.
Sanity check: am I correct in thinking those env vars do not need to change?
In spite of feeling like I was close to resolving everything necessary to use the put/get in my IT - and test locally, maybe I do indeed need to skip the local testing and just request installs in a production Galaxy instance.
There are such ITs, Juypter Notebook, and RStudio, you can test it on EU for example.
You need quotes afaik:
docker_run_extra_arguments: "--add-host localhost:host-gateway --platform linux/amd64"
Please share your entire job_conf file.
I was also told that IT work in this one GitHub - bgruening/docker-galaxy: 🐋📊📚 Docker Images tracking the stable Galaxy releases. - but I don’t want to sent you into this rabbit whole.
Adding the quotes causes the infinite “Waiting…” state. Removing the quotes allows the IT to start, almost immediately. But it still gives the “Could not connect to galaxy instance” when I attempt a “put”.
Here is a full example file: galaxy/config/job_conf.yml.interactivetools at 5d197246ff7d9718f9a7cc645663a2b087d4d872 · galaxyproject/galaxy · GitHub
What you can try is to start everything but then dive into the container, try to ping your host Galaxy. Execute the get/put from inside the container and try if you see more debug errors. I would recommend to do this with Jupyter where you know it should be working.
In the job_conf.yml, do I keep this:
execution:
default: docker_dispatch
or should it change to default: docker_local
I will just begin experimenting… Thanks for your patience.
In the .yml template that you linked to, it has:
#docker_run_extra_arguments: --add-host localhost:host-gateway
When I uncomment that line and run, I’m once again stuck in the “Waiting…” state for the IT. But if I append --platform linux/amd64 to that line and re-run, the IT enters the “Open…” state almost immediately. And a put results in the usual Exception: Could not connect to a galaxy instance
I’ve continued to try several attempts - each trying to tweak some small thing; confirmed my API key matches, and I’ve examined log files, e.g.,
gunicorn.log:
galaxy.tool_util.output_checker INFO 2025-05-23 18:30:59,341 [pN:main.1,p:20384,tN:LocalRunner.work_thread-2] Job error detected, failing job. Reasons are [{‘type’: ‘exit_code’, ‘desc’: ‘Fatal error: Exit code 12 ()’, ‘exit_code’: 12, ‘code_desc’: ‘’, ‘error_level’: 3}]
yet I continue to get the “Could not connect to galaxy instance” when I attempt a “put” from my IT.
How can I find out where that “Could not connect…” msg is coming from. I assumed it was in the bioblend code (or Galaxy itself), but I could not find it doing a global search.
EDIT: found it - inside the get_galaxy_connection function in the galaxy_ie_helpers/init.py
Is there something I’m missing about having Galaxy start my IT/container that drops me into a bash shell, from which I could try to ping the Galaxy host?
Here’s is my entire job_conf.yml:
(base) heiland@yoga:~/dev/galaxy-24.2.3$ more config/job_conf.yml
## A sample job config for InteractiveTools using local runner. ##
runners:
local:
load: galaxy.jobs.runners.local:LocalJobRunner
workers: 4
# Uncomment if dynamic handlers are defined in "gravity:handlers" section in galaxy.yml
#
#handling:
# assign:
# - db-skip-locked
execution:
default: docker_dispatch
environments:
local:
runner: local
docker_local:
runner: local
docker_enabled: true
#docker_volumes: $defaults,/mnt/galaxyData/libraries:ro,/mnt/galaxyData/indices:ro
#docker_volumes_from: parent_container_name
#docker_memory: 24G
#docker_sudo: false
#docker_sudo_cmd: /usr/bin/sudo -extra_param
#docker_net: bridge
#docker_auto_rm: true
#docker_set_user: $UID
docker_set_user:
# InteractiveTools do need real hostnames or URLs to work - simply specifying IPs will not work.
# If you develop interactive tools on your 'localhost' and don't have a proper domain name
# you need to tell all Docker containers a hostname where Galaxy is running.
# This can be done via the add-host parameter during the `docker run` command.
# 'localhost' here is an arbitrary hostname that matches the IP address of your
# Galaxy host. Make sure this hostname ('localhost') is also set in your galaxy.yml file, e.g.
# `galaxy_infrastructure_url: http://localhost:8080`.
#docker_run_extra_arguments: add-host localhost:host-gateway
docker_run_extra_arguments: --add-host localhost:host-gateway --platform linux/amd64
#docker_cmd: /usr/local/custom_docker/docker
#docker_host:
#docker_container_id_override: busybox:1.36.1-glibc
#docker_default_container_id: busybox:1.36.1-glibc
#require_container: true
#container_monitor: true
#container_monitor_result: file
#container_monitor_command: python /path/to/galaxy/lib/galaxy_ext/container_monitor/monitor.py
#container_monitor_get_ip_method: null
#container_resolvers_config_file: null
#container_resolvers:
docker_dispatch:
runner: dynamic
type: docker_dispatch
docker_destination_id: docker_local
default_destination_id: local
- running Jupyter notebook on usegalaxy.eu - works fine! In addition to doing a “put”, I explicitly perform some of the steps in the “put” method:
!echo "foo" >foo.txt
# put("foo.txt")
print('HISTORY_ID= ',os.environ['HISTORY_ID'])
print('API_KEY= ',os.environ['API_KEY'])
print('GALAXY_URL= ',os.environ['GALAXY_URL'])
# print('DOCKER_HOST= ',os.environ['DOCKER_HOST'])
print('GALAXY_WEB_PORT= ',os.environ['GALAXY_WEB_PORT'])
history_id = None or os.environ['HISTORY_ID']
print("history_id= ",history_id)
gi = get_galaxy_connection(history_id=history_id)
print("gi= ",gi)
→
HISTORY_ID= fbc2390c374706b3
API_KEY= 2cf2-blargh
GALAXY_URL= https://usegalaxy.eu
GALAXY_WEB_PORT= 8080
history_id= fbc2390c374706b3
gi= <bioblend.galaxy.objects.galaxy_instance.GalaxyInstance object at 0x7fd884a9f050>
However, trying to do the very same thing on my local Galaxy server generates the error:
Running a local Galaxy server on Ubuntu and running the Jupyter Notebook IT, I copy/paste a modified version (with debug prints) of galaxy_ie_helpers __init__.py into the notebook’s cell and execute that cell so that I now have my own “put” and associated methods. When I create a “foo.txt” and attempt to “put” it with my local version, I get the following debug prints:
get_galaxy_connection: 1) key=7c53...
get_galaxy_connection: url=http://127.0.0.1:8080
_test_url: url=%s obj=%s http://127.0.0.1:8080 True
_test_url: obj branch: key=7c53..., gi=<bioblend.galaxy.objects.galaxy_instance.GalaxyInstance object at 0x74491407fd00>
TestURL url=%s state=failure http://127.0.0.1:8080
get_galaxy_connection: _test_url returned None
get_galaxy_connection: app_path(1) http://127.0.0.1:8080
get_galaxy_connection: app_path(2)
get_galaxy_connection: GALAXY_WEB_PORT = 8080
get_galaxy_connection: built_galaxy_url = http://b'172.17.0.1':8080/ <---WHY??
get_galaxy_connection: url = http://b'172.17.0.1':8080
_test_url: url=%s obj=%s http://b'172.17.0.1':8080 True
_test_url: obj branch: key=7c53..., gi=<bioblend.galaxy.objects.galaxy_instance.GalaxyInstance object at 0x744914925970>
TestURL url=%s state=failure http://b'172.17.0.1':8080
get_galaxy_connection: final gi= None
I’m confused why I’m seeing built_galaxy_url = http://b'172.17.0.1':8080/. It came from the _get_ip() which says it returns the “IP address for the docker host”.
I want to repeat this same debug code on the official Jupyter notebook IT
on usegalaxy.eu, but it seems to be taking forever to startup the IT.
Finally got the Jupyter notebook working on usegalaxy.eu and, using my modified galaxy_ie_helpers code with debug prints, I get the following when doing a “put”:
get_galaxy_connection: 1) key=2cf2...
get_galaxy_connection: url=https://usegalaxy.eu
_test_url: url=%s obj=%s https://usegalaxy.eu True
_test_url: obj branch: key=2cf2..., gi=<bioblend.galaxy.objects.galaxy_instance.GalaxyInstance object at 0x7f981c1142f0>
get_galaxy_connection: _test_url returned <bioblend.galaxy.objects.galaxy_instance.GalaxyInstance object at 0x7f981c1142f0>
compared to the debug prints when running a local server:
get_galaxy_connection: 1) key=7c53...
get_galaxy_connection: url=http://127.0.0.1:8080
_test_url: url=%s obj=%s http://127.0.0.1:8080 True
_test_url: obj branch: key=7c53..., gi=<bioblend.galaxy.objects.galaxy_instance.GalaxyInstance object at 0x74491407fd00>
TestURL url=%s state=failure http://127.0.0.1:8080 <-- differs!
get_galaxy_connection: _test_url returned None
get_galaxy_connection: app_path(1) http://127.0.0.1:8080
get_galaxy_connection: app_path(2)
get_galaxy_connection: GALAXY_WEB_PORT = 8080
get_galaxy_connection: built_galaxy_url = http://b'172.17.0.1':8080/ <---WHY??
get_galaxy_connection: url = http://b'172.17.0.1':8080
_test_url: url=%s obj=%s http://b'172.17.0.1':8080 True
_test_url: obj branch: key=7c53..., gi=<bioblend.galaxy.objects.galaxy_instance.GalaxyInstance object at 0x744914925970>
TestURL url=%s state=failure http://b'172.17.0.1':8080
get_galaxy_connection: final gi= None
Inserting a traceback in _test_url:
except Exception:
import traceback
print(traceback.format_exc())
provides:
get_galaxy_connection: 1) key=7c53...
get_galaxy_connection: url=http://127.0.0.1:8080
_test_url: url=%s obj=%s http://127.0.0.1:8080 True
_test_url: obj branch: key=7c53..., gi=<bioblend.galaxy.objects.galaxy_instance.GalaxyInstance object at 0x74ba6c7254f0>
Traceback (most recent call last):
File "<ipython-input-4-85bc6f23efe6>", line 41, in _test_url
gi.histories.get(history_id)
File "/opt/conda/lib/python3.8/site-packages/bioblend/galaxy/objects/client.py", line 192, in get
return self._get_container(id_, wrappers.History)
File "/opt/conda/lib/python3.8/site-packages/bioblend/galaxy/objects/client.py", line 88, in _get_container
res = show_f(id_)
File "/opt/conda/lib/python3.8/site-packages/bioblend/galaxy/histories/__init__.py", line 188, in show_history
return self._get(id=history_id, contents=contents, params=params)
File "/opt/conda/lib/python3.8/site-packages/bioblend/galaxy/client.py", line 149, in _get
raise ConnectionError(msg, body=r.text,
bioblend.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=8080):
Max retries exceeded with url: /api/histories/f597429621d6eb2b?key=7c53...
(Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x74ba6c6eaf70>:
Failed to establish a new connection: [Errno 111] Connection refused')), 0 attempts left:
When I google for the HTTPConnection error, I see this (among other things) - whether or not it’s relevant:
Docker networking issues:
If running in a Docker container, ensure proper network configuration
so containers can communicate with each other, or try running with --net=host.
I need to move on. I’ve spent too much time trying to debug why I can’t test my IT on a local Galaxy instance. If possible, I’d like to request the latest version be installed for testing on a production server I guess. Is this possible @bjoern.gruening ? I’ve updated my TTS version at Galaxy Tool Shed . Please let me know if I need to create a repo at the (non-test) toolshed, or other actions that are appropriate. Thanks.
Actually, I seem to have issues with creating a repo on the main toolshed and I suspect it has to do with accidental duplicate usernames (“rheiland”) and two different emails for accounts - planemo shed_create fails on testtoolshed - #5 by rheiland
Thanks. I’ve done that - interactivetool_pcstudio.xml for SMB conf by rheiland · Pull Request #287 · usegalaxy-eu/galaxy · GitHub
I tried testing on my Ubuntu with my fork of that repo, but a sh run.sh eventually gave me ERROR: No matching distribution found for social-auth-core==4.6.1
Returning to the “get” function in “galaxy_ie_helpers”, can someone explain what I’m doing wrong here?
It seems to be related to repeating a “get” with the same ID as previously.
HI @rheiland
I’m not familiar with the IE setup, so @bjoern.gruening can continue with that, but for the commands, are you running the gx_get() command? This topic covers how it works:
Maybe review and explain what you are doing differently?
No, I’m not running gx_get(). Isn’t that specific to R? My IT is Python-based, hence my attempt at using the get(), put() in the galaxy_ie_helpers module.