Hi again, sorry to use the ping, but we would love to get the put() and get() working @hexylena
Could the issue be related to some missing docker options? We have this currently in our job_conf.xml
The container has a DEBUG option, I believe, I should have thought of that sooner. If you set that environment variable, then the get/put commands will print out additional logging information that might help you diagnose the issue.
I think you’ll need to set that in the tool xml.
Try adding something like <environment_variable name="DEBUG">True</environment_variable>
xref galaxy_ie_helpers/galaxy_ie_helpers at master · bgruening/galaxy_ie_helpers · GitHubinit.py#L12
Yes thanks, did that actually manually in the container. But where is actually this logged to? I.e. where should I be able to find the log file?
it should log inside the notebook, whenever you run the command. If not, you can try running the get command in a terminal inside jupyter
Right, I see no logging output at all in the notebook, only the exception.
In the running container I did:
root@f0557899417c:/storage/galaxy/jobs_directory/001/1252/working/jupyter# echo $DEBUG
true
In addition I manually edited the init.py :
#!/usr/bin/env python
from bioblend.galaxy import objects
from bioblend.galaxy import GalaxyInstance
from bioblend.galaxy.histories import HistoryClient
from bioblend.galaxy.datasets import DatasetClient
import subprocess
import argparse
import re
import os
from string import Template
import logging
DEBUG = os.environ.get('DEBUG', "False").lower() == 'true'
#if DEBUG:
# logging.basicConfig(level=logging.DEBUG)
#logging.getLogger("bioblend").setLevel(logging.CRITICAL)
logging.basicConfig(level=logging.DEBUG)
log = logging.getLogger()
And I have added some more log.debug messages in the code. I see no trace of the logging output in the notebook though.
This is how it looks, you see the printout of the code where I added log.debug message, but the logging message is not actually shown in the notebook:
try put -h in the terminal
1 Like
yes, sorry, realized. We get debug output now in the terminal, thanks!
1 Like
So this is the reason:
root@f0557899417c:/storage/galaxy/jobs_directory/001/1252/working/jupyter# put --history-id 63cd3858d057a6d1 -p /storage/galaxy/jobs_directory/001/1252/working/jupyter/testfile.txt
DEBUG:root:here I am
DEBUG:root:Host IP determined to be b'172.17.0.1\n'
DEBUG:root:TestURL url=https://galaxy-hepp.hpc.uio.no obj=True
DEBUG:bioblend:GET - attempts left: 1; retry delay: 10
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): galaxy-hepp.hpc.uio.no:443
ERROR:bioblend:HTTPSConnectionPool(host='galaxy-hepp.hpc.uio.no', port=443): Max retries exceeded with url: /api/histories/63cd3858d057a6d1 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1125)'))), 0 attempts left
DEBUG:root:TestURL url=https://galaxy-hepp.hpc.uio.no state=failure
DEBUG:root:TestURL url=http://b'172.17.0.1':8080 obj=True
DEBUG:bioblend:GET - attempts left: 1; retry delay: 10
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): b'172.17.0.1':8080
ERROR:bioblend:HTTPConnectionPool(host="b'172.17.0.1'", port=8080): Max retries exceeded with url: /api/histories/63cd3858d057a6d1 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f90c336bbe0>: Failed to establish a new connection: [Errno -2] Name or service not known')), 0 attempts left
DEBUG:root:TestURL url=http://b'172.17.0.1':8080 state=failure
DEBUG:root:do I see this?
Traceback (most recent call last):
File "/opt/conda/bin/put", line 17, in <module>
put( args.filepath, file_type=args.filetype, history_id=args.history_id )
File "/opt/conda/lib/python3.8/site-packages/galaxy_ie_helpers/__init__.py", line 115, in put
gi = get_galaxy_connection(history_id=history_id)
File "/opt/conda/lib/python3.8/site-packages/galaxy_ie_helpers/__init__.py", line 101, in get_galaxy_connection
raise Exception(msg)
Exception: Could not connect to a galaxy instance. Please contact your SysAdmin for help with this error
So then to figure out how to fix that.
Could it be that the CA we use:
issuer=C = NL, O = GEANT Vereniging, CN = GEANT OV RSA CA 4
needs to be included in /etc/ssl/certs in the container?
I do not see it there.
We have solved the put() error - the problem was due to the lack of certificate cocatination on our galaxy-hepp.uio.no server, so the CA certificate was not available for requests.
Now in place and put() does not give that error anymore.
(Another error is being dealt with related to permissions in /storage/galaxy/tmp - but that is another issue).
2 Likes