Database upgrade error - 22.05 to 24.01

We needed to upgrade from 22.05 to 24.01 to support the new Slurm version.
sh manage_db.sh upgrade fails. If I could find the SQL lines being executed, I would try to do so manually. But they appear to be buried inside layers of objects. Here’s the full message:

sh manage_db.sh upgrade
Activating virtualenv at .venv
INFO:alembic.runtime.migration:Context impl SQLiteImpl.
INFO:alembic.runtime.migration:Will assume non-transactional DDL.
INFO:alembic.runtime.migration:Running upgrade c63848676caf → 04288b6a5b25, make dataset uuids unique
Traceback (most recent call last):
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py”, line 1967, in _exec_single_context
self.dialect.do_execute(
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/.venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py”, line 924, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: near “FROM”: syntax error

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/./scripts/db.py”, line 37, in
main()
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/./scripts/db.py”, line 33, in main
args.func(args)
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/lib/galaxy/model/migrations/base.py”, line 265, in upgrade
self._exec_command(“upgrade”, args)
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/lib/galaxy/model/migrations/base.py”, line 289, in _exec_command
getattr(dbscript, command)(args)
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/lib/galaxy/model/migrations/base.py”, line 202, in upgrade
self._upgrade_to_head(args.sql)
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/lib/galaxy/model/migrations/dbscript.py”, line 88, in _upgrade_to_head
self._upgrade_to_revision(“gxy@head”, is_sql_mode)
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/lib/galaxy/model/migrations/base.py”, line 234, in _upgrade_to_revision
command.upgrade(self.alembic_config, rev, is_sql_mode)
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/.venv/lib/python3.10/site-packages/alembic/command.py”, line 403, in upgrade
script.run_env()
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/.venv/lib/python3.10/site-packages/alembic/script/base.py”, line 583, in run_env
util.load_python_file(self.dir, “env.py”)
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/.venv/lib/python3.10/site-packages/alembic/util/pyfiles.py”, line 95, in load_python_file
module = load_module_py(module_id, path)
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/.venv/lib/python3.10/site-packages/alembic/util/pyfiles.py”, line 113, in load_module_py
spec.loader.exec_module(module) # type: ignore
File “”, line 883, in exec_module
File “”, line 241, in _call_with_frames_removed
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/lib/galaxy/model/migrations/alembic/env.py”, line 146, in
run_migrations_online()
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/lib/galaxy/model/migrations/alembic/env.py”, line 40, in run_migrations_online
_configure_and_run_migrations_online(url)
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/lib/galaxy/model/migrations/alembic/env.py”, line 123, in _configure_and_run_migrations_online
context.run_migrations()
File “”, line 8, in run_migrations
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/.venv/lib/python3.10/site-packages/alembic/runtime/environment.py”, line 948, in run_migrations
self.get_context().run_migrations(**kw)
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/.venv/lib/python3.10/site-packages/alembic/runtime/migration.py”, line 627, in run_migrations
step.migration_fn(**kw)
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/lib/galaxy/model/migrations/alembic/versions_gxy/04288b6a5b25_make_dataset_uuids_unique.py”, line 73, in upgrade
_update_dataset_associations_to_point_to_latest_active_datasets(connection)
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/lib/galaxy/model/migrations/alembic/versions_gxy/04288b6a5b25_make_dataset_uuids_unique.py”, line 292, in _update_dataset_associations_to_point_to_latest_active_datasets
connection.execute(update_hda_links)
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py”, line 1418, in execute
return meth(
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/.venv/lib/python3.10/site-packages/sqlalchemy/sql/elements.py”, line 515, in _execute_on_connection
return connection._execute_clauseelement(
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py”, line 1640, in _execute_clauseelement
ret = self._execute_context(
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py”, line 1846, in _execute_context
return self._exec_single_context(
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py”, line 1986, in _exec_single_context
self._handle_dbapi_exception(
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py”, line 2353, in _handle_dbapi_exception
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/.venv/lib/python3.10/site-packages/sqlalchemy/engine/base.py”, line 1967, in _exec_single_context
self.dialect.do_execute(
File “/common/compbiomed-galaxy/galaxy/24.01/galaxy/.venv/lib/python3.10/site-packages/sqlalchemy/engine/default.py”, line 924, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) near “FROM”: syntax error
[SQL:
UPDATE history_dataset_association
SET dataset_id=t.to_dataset_id
FROM temp_active_mapping t
WHERE t.from_dataset_id = dataset_id
]
(Background on this error at: Error Messages — SQLAlchemy 2.0 Documentation)

What’s my next step? The URL at the end of the message was no help.

Welcome @caldodge

Your upgrade is making a huge jump between releases and the “FROM” statement seems to be having trouble connecting between the tables changes between them.

To migrate between releases, and to incorporate the changes introduced in each, we recommend upgrading one release at a time. This way the changes for each are applied before the next change is applied. Apologies if this is confusing language! In short: upgrade to each release sequentially until you are at the current stable release.

Painful, but it is the supported way to do this. Let us know if you need help as you work through this. We’ve had other upgrade out of the 22.NN releases and there were a few hiccups but these were able to be resolved. :slight_smile:

XREf →