Unable to upgrade Galaxy from 16.07

1

Hi,

Hope I can find help here. I have a Galaxy server that was purchased a number of years ago from Bioteam (Slipstream Appliance). They administered it up to version 16.07 but then discontinued their support. I need to update to the latest Galaxy version using their instructions supplemented with instructions here - Galaxy Community Hub . I am at a beginner level of dealing with Linux servers (why I purchased a supported setup) so forgive my ignorance.

Setup:

Bioteam Appliance (Galaxy preloaded Linux server)

Ubuntu 14.04.6 LTS (Trusty Tahr)

PostgreSQL 9.3 database server

Galaxy 16.07

Current approach to upgrade:

  1. Login, become root, and take a ZFS snapshot to enable rollback if problem.

sudo –I

cd /slipstream

supervisorctl stop all && service postgresql stop

zfs snapshot slipstream@pre_upgrade_from_galaxy16.07_2021-11-13

service postgresql start && supervisorctl start all

  1. Fetch new version using git, update virtualenv, and migrate DB

cd /slipstream/galaxy/production/galaxy-dist

git fetch origin && git checkout release_17.01 && git pull --ff-only origin release_17.01

./scripts/common_startup.sh

reboot

sh manage_db.sh upgrade

git status

sh ./scripts/migrate_tools/0011_tools.sh install_dependencies

service postgresql start && supervisorctl start all

  1. The screen output I get with the last command is:
  • Starting PostgreSQL 9.3 database server [ OK ]

handler1: ERROR (abnormal termination)

web0: ERROR (abnormal termination)

web1: ERROR (abnormal termination)

web5: ERROR (abnormal termination)

web4: ERROR (abnormal termination)

handler0: ERROR (abnormal termination)

web2: ERROR (abnormal termination)

web3: ERROR (abnormal termination)

There are other screen outputs from the prior commands that I could also share. Or can look for specific issues.

I looked at the log files – comparing log files from startup before upgrade and after upgrade. One thing I noticed is that before upgrade it loads up to database version 135 (and indicates that the version is 135). But after the upgrade it only loads up to version 133 – and then checks for version and says that the database version is 135. Could this by related to why it won’t start??

Any suggestions would be greatly appreciated.

Thanks
Michael

2

First guess would be to make sure that your database (postgresql) is running when you execute sh manage_db.sh upgrade. Can you verify that, and provide the output?

3

Sure!

root@wsu-appliance:/slipstream/home/michael# service postgresql start

  • Starting PostgreSQL 9.3 database server [ OK ]
    root@wsu-appliance:/slipstream/home/michael# sh manage_db.sh upgrade
    sh: 0: Can’t open manage_db.sh
    root@wsu-appliance:/slipstream/home/michael# cd /slipstream/galaxy/production/ga laxy-dist
    root@wsu-appliance:/slipstream/galaxy/production/galaxy-dist# service postgresql start
  • Starting PostgreSQL 9.3 database server [ OK ]
    root@wsu-appliance:/slipstream/galaxy/production/galaxy-dist# sh manage_db.sh up grade
    Activating virtualenv at .venv
    Traceback (most recent call last):
    File “./scripts/manage_db.py”, line 23, in
    invoke_migrate_main()
    File “./scripts/manage_db.py”, line 19, in invoke_migrate_main
    main( repository=repo, url=db_url )
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/migrate/versioning/shell.py”, line 209, in main
    ret = command_func(**kwargs)
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/migrate/versioning/api.py”, line 186, in upgrade
    return _migrate(url, repository, version, upgrade=True, err=err, **opts)
    File “”, line 2, in _migrate
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/migrate/versioning/util/init.py”, line 160, in with_engine
    return f(*a, **kw)
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/migrate/versioning/api.py”, line 342, in _migrate
    schema = ControlledSchema(engine, repository)
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/migrate/versioning/schema.py”, line 33, in init
    self.load()
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/migrate/versioning/schema.py”, line 54, in load
    exceptions.DatabaseNotControlledError(str(exc)), tb)
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/migrate/versioning/schema.py”, line 45, in load
    self.table = Table(tname, self.meta, autoload=True)
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/sqlalchemy/sql/schema.py”, line 440, in new
    metadata._remove_table(name, schema)
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/sqlalchemy/util/langhelpers.py”, line 60, in exit
    compat.reraise(exc_type, exc_value, exc_tb)
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/sqlalchemy/sql/schema.py”, line 435, in new
    table._init(name, metadata, *args, **kw)
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/sqlalchemy/sql/schema.py”, line 510, in _init
    self.autoload(metadata, autoload_with, include_columns)
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/sqlalchemy/sql/schema.py”, line 534, in autoload
    self, include_columns, exclude_columns
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/sqlalchemy/engine/base.py”, line 1972, in run_callable
    return conn.run_callable(callable    , *args, **kwargs)
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/sqlalchemy/engine/base.py”, line 1477, in run_callable
    return callable    (self, *args, **kwargs)
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/sqlalchemy/engine/default.py”, line 364, in reflecttable
    return insp.reflecttable(table, include_columns, exclude_columns)
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/sqlalchemy/engine/reflection.py”, line 571, in reflecttable
    raise exc.NoSuchTableError(table.name)
    migrate.exceptions.DatabaseNotControlledError: migrate_version
    root@wsu-appliance:/slipstream/galaxy/production/galaxy-dist#
4

Can you try to explicitly set the galaxy config file when trying the update? Your version may still be using an ini file, so something like this:

sh manage_db.sh upgrade -c /path/to/galaxy.ini
5

Sure. the *.ini is actually still universe_wsgi.ini
Here is the output from this:

root@wsu-appliance:/slipstream/galaxy/production/galaxy-dist# sh manage_db.sh upgrade -c /slipstream/galaxy/production/config/universe_wsgi.ini
Activating virtualenv at .venv
Traceback (most recent call last):
File “./scripts/manage_db.py”, line 23, in
invoke_migrate_main()
File “./scripts/manage_db.py”, line 19, in invoke_migrate_main
main( repository=repo, url=db_url )
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/migrate/versioning/shell.py”, line 209, in main
ret = command_func(**kwargs)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/migrate/versioning/api.py”, line 186, in upgrade
return _migrate(url, repository, version, upgrade=True, err=err, **opts)
File “”, line 2, in _migrate
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/migrate/versioning/util/init.py”, line 160, in with_engine
return f(*a, **kw)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/migrate/versioning/api.py”, line 342, in _migrate
schema = ControlledSchema(engine, repository)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/migrate/versioning/schema.py”, line 33, in init
self.load()
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/migrate/versioning/schema.py”, line 54, in load
exceptions.DatabaseNotControlledError(str(exc)), tb)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/migrate/versioning/schema.py”, line 45, in load
self.table = Table(tname, self.meta, autoload=True)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py”, line 440, in new
metadata._remove_table(name, schema)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py”, line 60, in exit
compat.reraise(exc_type, exc_value, exc_tb)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py”, line 435, in new
table._init(name, metadata, *args, **kw)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py”, line 510, in _init
self._autoload(metadata, autoload_with, include_columns)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/sql/schema.py”, line 534, in _autoload
self, include_columns, exclude_columns
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py”, line 1971, in run_callable
with self.contextual_connect() as conn:
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py”, line 2039, in contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py”, line 2078, in _wrap_pool_connect
e, dialect, self)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py”, line 1405, in _handle_dbapi_exception_noconnection
exc_info
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py”, line 202, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py”, line 2074, in _wrap_pool_connect
return fn()
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py”, line 376, in connect
return _ConnectionFairy._checkout(self)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py”, line 713, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py”, line 480, in checkout
rec = pool._do_get()
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py”, line 1060, in _do_get
self._dec_overflow()
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/util/langhelpers.py”, line 60, in exit
compat.reraise(exc_type, exc_value, exc_tb)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py”, line 1057, in _do_get
return self._create_connection()
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py”, line 323, in _create_connection
return _ConnectionRecord(self)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py”, line 449, in init
self.connection = self.__connect()
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/pool.py”, line 607, in __connect
connection = self.__pool._invoke_creator(self)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py”, line 97, in connect
return dialect.connect(*cargs, **cparams)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py”, line 385, in connect
return self.dbapi.connect(*cargs, **cparams)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/psycopg2/init.py”, line 164, in connect
conn = _connect(dsn, connection_factory=connection_factory, async=async)
migrate.exceptions.DatabaseNotControlledError: (psycopg2.OperationalError) FATAL: role “root” does not exist

6

migrate.exceptions.DatabaseNotControlledError: (psycopg2.OperationalError) FATAL: role “root” does

Try executing when logged in as the Galaxy user?

7

I tried logging in as “Galaxy” and as my user name - “Michael” :

root@wsu-appliance:/slipstream/galaxy/production/galaxy-dist# su galaxy
galaxy@wsu-appliance:~/production/galaxy-dist$ sh manage_db.sh upgrade -c /slipstream/galaxy/production/config/universe_wsgi.ini
Activating virtualenv at .venv
galaxy@wsu-appliance:~/production/galaxy-dist$ su michael
Password:
michael@wsu-appliance:/slipstream/galaxy/production/galaxy-dist$ sh manage_db.sh upgrade -c /slipstream/galaxy/production/config/universe_wsgi.ini
Activating virtualenv at .venv
Traceback (most recent call last):
File “./scripts/manage_db.py”, line 23, in
invoke_migrate_main()
File “./scripts/manage_db.py”, line 15, in invoke_migrate_main
config = get_config( sys.argv )
File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/model/orm/scripts.py”, line 93, in get_config
properties = load_app_properties( ini_file=config_file, config_prefix=config_override )
File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/util/properties.py”, line 49, in load_app_properties
with open(ini_file) as f:
IOError: [Errno 13] Permission denied: ‘/slipstream/galaxy/production/config/universe_wsgi.ini’
michael@wsu-appliance:/slipstream/galaxy/production/galaxy-dist$

8

These are the other users:

root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/usr/sbin/nologin
man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin
mail:x:8:8:mail:/var/mail:/usr/sbin/nologin
news:x:9:9:news:/var/spool/news:/usr/sbin/nologin
uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin
proxy:x:13:13:proxy:/bin:/usr/sbin/nologin
www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
backup:x:34:34:backup:/var/backups:/usr/sbin/nologin
list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin
irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin
gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin
nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin
libuuid:x:100:101::/var/lib/libuuid:/bin/sh
syslog:x:101:103::/home/syslog:/bin/false
messagebus:x:102:105::/var/run/dbus:/bin/false
sshd:x:103:65534::/var/run/sshd:/usr/sbin/nologin
postgres:x:104:110:PostgreSQL administrator,:/var/lib/postgresql:/bin/bash
slipstream:x:1005:1005:SlipStream-Management-Account:/slipstream/home/slipstream:/bin/bash
galaxy:x:1000:1000::/slipstream/galaxy:/bin/bash
postfix:x:105:111::/var/spool/postfix:/bin/false
sgeadmin:x:106:113::/var/lib/gridengine:/bin/false
slipuser:x:1006:1006:SlipStream-User-Account:/slipstream/home/slipuser:/bin/bash
michael:x:1007:1007::/slipstream/home/michael:/bin/bash

9

Was there any additional output after the activation of the .venv when running as ‘galaxy’? Your copy&paste goes right into you entering su michael.

10

No. It went to the prompt as shown without any hesitation.

11

Just in case I had done something that caused it to bounce out of virtualenv - I did a rollback to the previous snapshot and then repeated the whole update process except after the reboot I logged into “Galaxy” (not root) and this is what I got:

galaxy@wsu-appliance:~/production/galaxy-dist$ sh manage_db.sh upgrade -c /slips tream/galaxy/production/config/universe_wsgi.ini
Activating virtualenv at .venv
Traceback (most recent call last):
File “./scripts/manage_db.py”, line 23, in
invoke_migrate_main()
File “./scripts/manage_db.py”, line 19, in invoke_migrate_main
main( repository=repo, url=db_url )
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/migrate/versioning/shell.py”, line 209, in main
ret = command_func(**kwargs)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/migrate/versioning/api.py”, line 186, in upgrade
return _migrate(url, repository, version, upgrade=True, err=err, **opts)
File “”, line 2, in _migrate
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/migrate/versioning/util/init.py”, line 160, in with_engine
return f(*a, **kw)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/migrate/versioning/api.py”, line 345, in _migrate
changeset = schema.changeset(version)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/migrate/versioning/schema.py”, line 82, in changeset
changeset = self.repository.changeset(database, start_ver, version)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/migrate/versioning/repository.py”, line 225, in changeset
changes = [self.version(v).script(database, op) for v in versions]
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/migrate/versioning/repository.py”, line 189, in version
return self.versions.version(*p, **k)
File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site -packages/migrate/versioning/version.py”, line 163, in version
return self.versions[VerNum(vernum)]
KeyError: <VerNum(135)>
galaxy@wsu-appliance:~/production/galaxy-dist$ ^C

12

I forgot to say is that I also added: find . -name '*.pyc' -delete before “git fetch” - since I had seen this mentioned as a possible solution. When I didn’t add this I just got “Activating virtualenv at .venv” and nothing else.

So - the current procedure that got me to the last output was:

cd /slipstream/galaxy/production/galaxy-dist

find . -name '*.pyc' -delete

git fetch origin && git checkout release_17.01 && git pull --ff-only origin release_17.01

./scripts/common_startup.sh

reboot

su galaxy

cd /slipstream/galaxy/production/galaxy-dist

sh manage_db.sh upgrade -c /slipstream/galaxy/production/config/universe_wsgi.ini

13

Did you perhaps attempt to update to a Galaxy version greater than 17.01 before this? The maximum database version for 17.01 is 133, and it looks like your database might be at 135.

FWIIW, 17.09 has db version 135.

14

Yes. Initially I did try to update to 19.01 (latest version at the time) using the instructions left for me. It wouldn’t start so I rolled back using ZFS snapshot. I didn’t try upgrading again until recently - after getting advice here that I should try one version at a time - so I tried to upgrade to 17.01. Looking at the root history file though it does appear that there was a snapshot file pre- upgrade to 17.09 that was deleted. So I suspect that the previous administrator had upgraded to 17.09. Looking at my emails from them it does seem that the last version they upgraded to was 17.09. Not sure why the current version is 16.07. But it is always possible that that I did it (downgraded to 16.07??) at some point.

15

Thanks for this. I updated to 17.09 and now the database version and the migrate versions directory are now matched (135). I was then able to restart Galaxy.

I then tried to update to the next version - 18.01. The database migration worked fine. The tool config migration came up with an error:

sh ./scripts/migrate_tools/0011_tools.sh install_dependencies

File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/config.py”, line 7, in import ipaddress
ImportError: No module named ipaddress

And then it wouldn’t start (abnormal termination).

I noted in the release notes that there are quite a few changes including " The default web server used by Galaxy has changed from Paste to uWSGI and the default configuration file for Galaxy is now config/galaxy.yml instead of config/galaxy.ini ." The config file here is still universe_wsgi.ini. So - somehow the universe_wsgi.ini settings need to be moved to the sample galaxy.yml file?? There was also mention of moving settings from the xml to the new xml.sample files?? Wow. This is not an easy upgrade process it seems.

I looked for tutorials in server administration - but they only refer to newer versions (20 plus) and mainly refer to using “Ansible”.

Is there any resource to guide upgrades from 17.09??

16

Since last Monday I successfully migrated from “universe_wsgi.ini” to “galaxy.ini”; and also updated the “tool_conf.xml” , “tool_data_table_conf.xml” and “datatypes_conf.xml” files using the .sample files. After rebooting Galaxy started fine. The current version is 17_09 and database 35.

I then tried to update to the next version - 18_01 with the following outcome:

git fetch origin && git checkout release_18.01 && git pull --ff-only origin release_18.01
./scripts/common_startup.sh

Screen output:
Activating virtualenv at .venv
Requirement already satisfied: pip>=8.1 in ./.venv/lib/python2.7/site-packages
/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see Advanced Usage - urllib3 2.0.0.dev0 documentation.

  • SNIMissingWarning*
    /slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see Advanced Usage - urllib3 2.0.0.dev0 documentation.
  • InsecurePlatformWarning*
    Certificate did not match expected hostname: pypi.org. Certificate: {‘notAfter’: ‘Nov 23 18:41:10 2022 GMT’, ‘subjectAltName’: ((‘DNS’, ‘www.python.org’), (‘DNS’, '.python.org’), (‘DNS’, ‘docs.python.org’), (‘DNS’, ‘downloads.python.org’), (‘DNS’, ‘pypi.python.org’)), ‘subject’: (((‘commonName’, u’www.python.org’),),)}*
    Requirement already satisfied: numpy==1.9.2 in ./.venv/lib/python2.7/site-packages (from -r requirements.txt (line 4))
    Requirement already satisfied: bx-python==0.7.3 in ./.venv/lib/python2.7/site-packages (from -r requirements.txt (line 5))
    Collecting MarkupSafe==1.0 (from -r requirements.txt (line 6))
    /slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see Advanced Usage - urllib3 2.0.0.dev0 documentation.
  • SNIMissingWarning*
    /slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see Advanced Usage - urllib3 2.0.0.dev0 documentation.
  • InsecurePlatformWarning*
  • Could not fetch URL https://wheels.galaxyproject.org/simple/markupsafe/: There was a problem confirming the ssl certificate: [Errno 1] _ssl.c:510: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol - skipping*
    /slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see Advanced Usage - urllib3 2.0.0.dev0 documentation.
  • InsecurePlatformWarning*
  • Certificate did not match expected hostname: pypi.org. Certificate: {‘notAfter’: ‘Nov 23 18:41:10 2022 GMT’, ‘subjectAltName’: ((‘DNS’, ‘www.python.org’), (‘DNS’, '.python.org’), (‘DNS’, ‘docs.python.org’), (‘DNS’, ‘downloads.python.org’), (‘DNS’, ‘pypi.python.org’)), ‘subject’: (((‘commonName’, u’www.python.org’),),)}
  • Could not fetch URL https://pypi.python.org/simple/markupsafe/: There was a problem confirming the ssl certificate: hostname ‘pypi.org’ doesn’t match either of ‘www.python.org’, '.python.org’, ‘docs.python.org’, ‘downloads.python.org’, ‘pypi.python.org’ - skipping
  • Could not find a version that satisfies the requirement MarkupSafe==1.0 (from -r requirements.txt (line 6)) (from versions: )*
    No matching distribution found for MarkupSafe==1.0 (from -r requirements.txt (line 6))
    /slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/pip/vendor/requests/packages/urllib3/util/ssl.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see Advanced Usage - urllib3 2.0.0.dev0 documentation.
  • InsecurePlatformWarning*
    Certificate did not match expected hostname: pypi.org. Certificate: {‘notAfter’: ‘Nov 23 18:41:10 2022 GMT’, ‘subjectAltName’: ((‘DNS’, ‘www.python.org’), (‘DNS’, '.python.org’), (‘DNS’, ‘docs.python.org’), (‘DNS’, ‘downloads.python.org’), (‘DNS’, ‘pypi.python.org’)), ‘subject’: (((‘commonName’, u’www.python.org’),),)}*

reboot
sh manage_db.sh upgrade -c /slipstream/galaxy/production/config/galaxy.ini

Output:
Activating virtualenv at .venv
135 → 136…

Migration script for collections and workflows connections.

done
136 → 137…

Add copied_from_job_id column to jobs table

done
137 → 138…

Add version column to history_dataset_association table

done
138 → 139…

Migration script to add the history_dataset_association_history table.

Created history_dataset_association_history table
done
139 → 140…

Add dataset_version column to job_to_input_dataset table

done

sh ./scripts/migrate_tools/0011_tools.sh install_dependencies

Output:
Traceback (most recent call last):

  • File “./scripts/migrate_tools/migrate_tools.py”, line 17, in *
  • from tool_shed.galaxy_install.migrate.common import MigrateToolsApplication*
  • File “/slipstream/galaxy/production/galaxy-dist/lib/tool_shed/galaxy_install/migrate/common.py”, line 8, in *
  • import galaxy.config*
  • File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/config.py”, line 7, in *
  • import ipaddress*
    ImportError: No module named ipaddress

However galaxy then failed to start (abnormal termination) with the following Output from handler0.log:

Traceback (most recent call last):*

  • File “./scripts/paster.py”, line 27, in *
  • serve.run()*
  • File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/util/pastescript/serve.py”, line 1055, in run*
  • invoke(command, command_name, options, args[1:])*
  • File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/util/pastescript/serve.py”, line 1061, in invoke*
  • exit_code = runner.run(args)*
  • File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/util/pastescript/serve.py”, line 226, in run*
  • result = self.command()*
  • File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/util/pastescript/serve.py”, line 642, in command*
  • app = loadapp(app_spec, name=app_name, relative_to=base, global_conf=vars)*
  • File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py”, line 293, in loadapp*
  • return loadobj(APP, uri, name=name, *kw)
  • File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py”, line 318, in loadobj*
  • global_conf=global_conf)*
  • File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py”, line 343, in loadcontext*
  • global_conf=global_conf)*
  • File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py”, line 367, in _loadconfig*
  • return loader.get_context(object_type, name, global_conf)*
  • File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py”, line 508, in get_context*
  • section)*
  • File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py”, line 567, in _context_from_explicit*
  • value = import_string(found_expr)*
  • File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/util/pastescript/loadwsgi.py”, line 118, in import_string*
  • return pkg_resources.EntryPoint.parse(“x=” + s).load(False)*
  • File “/slipstream/galaxy/production/virtualenv/local/lib/python2.7/site-packages/pkg_resources/init.py”, line 2268, in load*
  • return self.resolve()*
  • File “/slipstream/galaxy/production/virtualenv/local/lib/python2.7/site-packages/pkg_resources/init.py”, line 2274, in resolve*
  • module = import(self.module_name, fromlist=[‘name’], level=0)*
  • File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/web/buildapp.py”, line 3, in *
  • from galaxy.webapps.galaxy.buildapp import app_factory*
  • File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/webapps/galaxy/buildapp.py”, line 12, in *
  • import galaxy.app*
  • File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/app.py”, line 18, in *
  • from galaxy.tools.cache import (*
  • File “/slipstream/galaxy/production/galaxy-dist/lib/galaxy/tools/init.py”, line 16, in *
  • import packaging.version*
    ImportError: No module named packaging.version

Any ideas on what to try next would be greatly appreciated!!

17

Looks like setuptools might be missing. You may want to try moving your .venv directory to a temporary name (e.g. .venv.old), and then start Galaxy and see if it will rebuild a working .venv.

18

Sure. I also provided the pip.log output. It looks like the problem is related to:“403 Client Error: SNI is required”

root@wsu-appliance:/slipstream/galaxy/production/galaxy-dist# mv .venv .venv.old
root@wsu-appliance:/slipstream/galaxy/production/galaxy-dist# cd .venv
bash: cd: .venv: No such file or directory
root@wsu-appliance:/slipstream/galaxy/production/galaxy-dist# ./scripts/common_startup.sh
Already using interpreter /usr/bin/python
New python executable in .venv/bin/python
Installing setuptools, pip…done.
Activating virtualenv at .venv
Downloading/unpacking pip>=8.1
Cannot fetch index base URL Simple index
Could not find any downloads that satisfy the requirement pip>=8.1
Cleaning up…
No distributions at all found for pip>=8.1
Storing debug log for failure in /root/.pip/pip.log

/slipstream/galaxy/production/galaxy-dist/.venv/bin/pip run on Tue Nov 30 14:09:55 2021
Downloading/unpacking pip>=8.1
Getting page Links for pip
Could not fetch URL https://pypi.python.org/simple/pip/: 403 Client Error: SNI is required
Will skip URL Links for pip when looking for download links for pip>=8.1
Getting page Simple index
Could not fetch URL https://pypi.python.org/simple/: 403 Client Error: SNI is required
Will skip URL Simple index when looking for download links for pip>=8.1
Cannot fetch index base URL Simple index
URLs to search for versions for pip>=8.1:

  • Links for pip
    Getting page Links for pip
    Could not fetch URL https://pypi.python.org/simple/pip/: 403 Client Error: SNI is required
    Will skip URL Links for pip when looking for download links for pip>=8.1
    Could not find any downloads that satisfy the requirement pip>=8.1
    Cleaning up…
    Removing temporary dir /slipstream/galaxy/production/galaxy-dist/.venv/build…
    No distributions at all found for pip>=8.1
    Exception information:
    Traceback (most recent call last):
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/pip/basecommand.py”, line 122, in main
    status = self.run(options, args)
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/pip/commands/install.py”, line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/pip/req.py”, line 1177, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
    File “/slipstream/galaxy/production/galaxy-dist/.venv/local/lib/python2.7/site-packages/pip/index.py”, line 277, in find_requirement
    raise DistributionNotFound(‘No distributions at all found for %s’ % req)
    DistributionNotFound: No distributions at all found for pip>=8.1
19

Just in case anyone reads this and is trying to upgrade Galaxy from pre-Galaxy 20 versions (in my case 16) with older pre-Ubuntu 20 versions (in my case 14) )— it is not possible since Galaxy and Ubuntu are co-dependent through this time period. Ubuntu is very difficult/impossible to upgrade successfully. Similarly Galaxy’s requirements (mainly Python) change greatly through this period. I ended up copying raw data off my Galaxy installation; wiping the server; installing Ubuntu 20; and then the Ansible version of Galaxy 21 – the latter based on these tutorials.

New install of Galaxy

Add on Ansible – install Proftpd

Ansible does make it easy - although there are a couple of errors and poorly explained sections in the tutorials that required some headscratching and tweaking.

Cross-posted topic: toolshed not working anymore with old galaxy?

1 Like