sqlite3.OperationalError: database is locked

Hello!

I updated galaxy from 20.09 to 21.05 and tried to invoke my workflow. But i got a message like this:

An error occurred
An error occurred while updating information with the server. Please contact a Galaxy administrator if the problem persists.
Ok Details
The following information can assist the developers in finding the source of the error:

{
  "userAgent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.77 Safari/537.36",
  "onLine": true,
  "version": "21.05",
  "xhr": {
    "readyState": 4,
    "responseText": "{\"err_msg\": \"Uncaught exception in exposed API method:\", \"err_code\": 0}",
    "responseJSON": {
      "err_msg": "Uncaught exception in exposed API method:",
      "err_code": 0
    },
    "status": 500,
    "statusText": "Internal Server Error"
  },
  "options": {
    "parse": true,
    "filters": {
      "update_time-ge": "1970-01-01T00:00:00.000Z",
      "visible": ""
    },
    "remove": false,
.....
.......
......

and was logged out of the server, and some tasks were not scheduled. When i tried to save the workflow in the workflow editor, i got similar error and also was logged out.

Galaxy.log says something like this:

galaxy.jobs.runners.local DEBUG 2021-10-04 15:23:37,549 [pN:main.web.1,p:514441,w:1,m:0,tN:LocalRunner.work_thread-8] (149) executing job script: /media/transgeneprep/DataStore/galaxy/database/jobs_directory/000/149/galaxy_149.sh
galaxy.web_stack.handlers INFO 2021-10-04 15:23:38,179 [pN:main.web.1,p:514441,w:1,m:0,tN:WorkflowRequestMonitor.monitor_thread] (Job[id=153,tool_id=star_fusion]) Handler '_default_' assigned using 'HANDLER_ASSIGNMENT_METHODS.DB_SKIP_LOCKED' assignment method
galaxy.tools.execute DEBUG 2021-10-04 15:23:38,432 [pN:main.web.1,p:514441,w:1,m:0,tN:WorkflowRequestMonitor.monitor_thread] Executed 1 job(s) for tool star_fusion request (1138.948 ms)
galaxy.workflow.run DEBUG 2021-10-04 15:23:39,276 [pN:main.web.1,p:514441,w:1,m:0,tN:WorkflowRequestMonitor.monitor_thread] Workflow step 106 of invocation 5 invoked (7473.255 ms)
galaxy.web.framework.decorators ERROR 2021-10-04 15:23:51,970 [pN:main.web.1,p:514441,w:1,m:0,tN:uWSGIWorker1Core3] Uncaught exception in exposed API method:
Traceback (most recent call last):
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:


Traceback (most recent call last):
  File "lib/galaxy/web/framework/decorators.py", line 184, in decorator
    rval = func(self, trans, *args, **kwargs)
  File "lib/galaxy/webapps/galaxy/api/datasets.py", line 157, in show
    return self.hda_serializer.serialize_to_view(dataset,
  File "lib/galaxy/managers/base.py", line 690, in serialize_to_view
    return self.serialize(item, all_keys, **context)
  File "lib/galaxy/managers/hdas.py", line 436, in serialize
    return super().serialize(hda, keys, user=user, **context)
  File "lib/galaxy/managers/datasets.py", line 606, in serialize
    serialized = super().serialize(dataset_assoc, keys, **context)
  File "lib/galaxy/managers/base.py", line 603, in serialize
    returned[key] = self.serializers[key](item, key, **context)
  File "lib/galaxy/managers/hdas.py", line 403, in <lambda>
    'resubmitted': lambda i, k, **c: self.hda_manager.has_been_resubmitted(i),
  File "lib/galaxy/managers/hdas.py", line 174, in has_been_resubmitted
    return self.app.model.context.query(query.exists()).scalar()
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3523, in scalar
    ret = self.one()
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3490, in one
    ret = self.one_or_none()
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3459, in one_or_none
    ret = list(self)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3535, in __iter__
    return self._execute_and_instances(context)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3560, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
    return meth(self, multiparams, params)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
    ret = self._execute_context(
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
    self._handle_dbapi_exception(
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
    util.raise_(
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked
[SQL: SELECT EXISTS (SELECT 1 
FROM job_to_output_dataset, job_state_history 
WHERE job_to_output_dataset.dataset_id = ? AND job_state_history.job_id = job_to_output_dataset.job_id AND job_state_history.state = ?) AS anon_1]
[parameters: (182, <states.RESUBMITTED: 'resubmitted'>)]
(Background on this error at: http://sqlalche.me/e/13/e3q8)
galaxy.workflow.run ERROR 2021-10-04 15:23:52,919 [pN:main.web.1,p:514441,w:1,m:0,tN:WorkflowRequestMonitor.monitor_thread] Failed to schedule Workflow[id=3,name=main.pipeline.PE.updated.24.08.2021 (imported from uploaded file)], problem occurred on WorkflowStep[index=21,type=tool].
Traceback (most recent call last):
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "lib/galaxy/workflow/run.py", line 193, in invoke
    workflow_invocation.steps.append(workflow_invocation_step)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 294, in __get__
    return self.impl.get(instance_state(instance), dict_)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/orm/attributes.py", line 730, in get
    value = self.callable_(state, passive)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/orm/strategies.py", line 759, in _load_for_state
    return self._emit_lazyload(
  File "<string>", line 1, in <lambda>
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/orm/strategies.py", line 900, in _emit_lazyload
    q(session)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/ext/baked.py", line 544, in all
    return list(self)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/ext/baked.py", line 444, in __iter__
    return q._execute_and_instances(context)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3560, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
    return meth(self, multiparams, params)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
    ret = self._execute_context(
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
    self._handle_dbapi_exception(
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
    util.raise_(
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked
[SQL: SELECT (SELECT workflow_invocation_to_subworkflow_invocation_association.subworkflow_invocation_id 
FROM workflow_invocation_to_subworkflow_invocation_association 
WHERE workflow_invocation_to_subworkflow_invocation_association.workflow_invocation_id = workflow_invocation_step.workflow_invocation_id AND workflow_invocation_to_subworkflow_invocation_association.workflow_step_id = workflow_invocation_step.workflow_step_id) AS anon_1, workflow_invocation_step.id AS workflow_invocation_step_id, workflow_invocation_step.create_time AS workflow_invocation_step_create_time, workflow_invocation_step.update_time AS workflow_invocation_step_update_time, workflow_invocation_step.workflow_invocation_id AS workflow_invocation_step_workflow_invocation_id, workflow_invocation_step.workflow_step_id AS workflow_invocation_step_workflow_step_id, workflow_invocation_step.state AS workflow_invocation_step_state, workflow_invocation_step.job_id AS workflow_invocation_step_job_id, workflow_invocation_step.implicit_collection_jobs_id AS workflow_invocation_step_implicit_collection_jobs_id, workflow_invocation_step.action AS workflow_invocation_step_action 
FROM workflow_invocation_step 
WHERE ? = workflow_invocation_step.workflow_invocation_id]
[parameters: (5,)]
(Background on this error at: http://sqlalche.me/e/13/e3q8)
galaxy.workflow.run ERROR 2021-10-04 15:23:54,931 [pN:main.web.1,p:514441,w:1,m:0,tN:WorkflowRequestMonitor.monitor_thread] Failed to execute scheduled workflow.
Traceback (most recent call last):
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "lib/galaxy/workflow/run.py", line 82, in __invoke
    outputs = invoker.invoke()
......
.....
.......

or this:

galaxy.jobs.handler ERROR 2021-10-04 15:41:35,174 [pN:main.web.1,p:514441,w:1,m:0,tN:JobHandlerStopQueue.monitor_thread] Exception in monitor_step
Traceback (most recent call last):
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "lib/galaxy/jobs/handler.py", line 927, in monitor
    self.monitor_step()
  File "lib/galaxy/jobs/handler.py", line 958, in monitor_step
    newly_deleted_jobs = self.sa_session.query(model.Job).enable_eagerloads(False) \
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3373, in all
    return list(self)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3535, in __iter__
    return self._execute_and_instances(context)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3560, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
    return meth(self, multiparams, params)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
    ret = self._execute_context(
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
    self._handle_dbapi_exception(
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
    util.raise_(
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked
[SQL: SELECT job.id AS job_id, job.create_time AS job_create_time, job.update_time AS job_update_time, job.history_id AS job_history_id, job.library_folder_id AS job_library_folder_id, job.tool_id AS job_tool_id, job.tool_version AS job_tool_version, job.galaxy_version AS job_galaxy_version, job.dynamic_tool_id AS job_dynamic_tool_id, job.state AS job_state, job.info AS job_info, job.copied_from_job_id AS job_copied_from_job_id, job.command_line AS job_command_line, job.dependencies AS job_dependencies, job.job_messages AS job_job_messages, job.param_filename AS job_param_filename, job.runner_name AS job_runner_name_1, job.job_stdout AS job_job_stdout, job.job_stderr AS job_job_stderr, job.tool_stdout AS job_tool_stdout, job.tool_stderr AS job_tool_stderr, job.exit_code AS job_exit_code, job.traceback AS job_traceback, job.session_id AS job_session_id, job.user_id AS job_user_id, job.job_runner_name AS job_job_runner_name, job.job_runner_external_id AS job_job_runner_external_id, job.destination_id AS job_destination_id, job.destination_params AS job_destination_params, job.object_store_id AS job_object_store_id, job.imported AS job_imported, job.params AS job_params, job.handler AS job_handler 
FROM job 
WHERE job.state IN (?, ?, ?) AND job.handler = ?]
[parameters: (<states.DELETED_NEW: 'deleted_new'>, <states.DELETING: 'deleting'>, <states.STOPPING: 'stop'>, 'main.web.1')]
(Background on this error at: http://sqlalche.me/e/13/e3q8)
galaxy.jobs.handler DEBUG 2021-10-04 15:41:35,268 [pN:main.web.1,p:514441,w:1,m:0,tN:WorkflowRequestMonitor.monitor_thread] Grabbing WorkflowInvocation failed (serialization failures are ok): (sqlite3.OperationalError) database is locked
[SQL: UPDATE workflow_invocation SET update_time=?, handler=? WHERE workflow_invocation.id IN (SELECT workflow_invocation.id 
FROM workflow_invocation 
WHERE workflow_invocation.handler IN (?, NULL) AND workflow_invocation.state = ? ORDER BY workflow_invocation.id)]
[parameters: ('2021-10-04 12:41:30.263975', 'main.web.1', '_default_', 'new')]
(Background on this error at: http://sqlalche.me/e/13/e3q8)
sqlalchemy.pool.impl.NullPool ERROR 2021-10-04 15:41:45,073 [pN:main.web.1,p:514441,w:1,m:0,tN:JobHandlerQueue.monitor_thread] Exception during reset or similar
Traceback (most recent call last):
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 697, in _finalize_fairy
    fairy._reset(pool)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 893, in _reset
    pool._dialect.do_rollback(self)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 558, in do_rollback
    dbapi_connection.rollback()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 140102957258496 and this is thread id 140102974043904.
sqlalchemy.pool.impl.NullPool ERROR 2021-10-04 15:41:45,073 [pN:main.web.1,p:514441,w:1,m:0,tN:JobHandlerQueue.monitor_thread] Exception closing connection <sqlite3.Connection object at 0x7f6c64150300>
Traceback (most recent call last):
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 697, in _finalize_fairy
    fairy._reset(pool)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 893, in _reset
    pool._dialect.do_rollback(self)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 558, in do_rollback
    dbapi_connection.rollback()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 140102957258496 and this is thread id 140102974043904.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 270, in _close_connection
    self._dialect.do_close(connection)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 564, in do_close
    dbapi_connection.close()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 140102957258496 and this is thread id 140102974043904.
galaxy.webapps.galaxy.controllers.user DEBUG 2021-10-04 15:41:45,433 [pN:main.web.1,p:514441,w:1,m:0,tN:uWSGIWorker1Core2] trans.app.config.auth_config_file: /media/transgeneprep/DataStore/galaxy/config/auth_conf.xml
galaxy.auth.providers.localdb DEBUG 2021-10-04 15:41:45,466 [pN:main.web.1,p:514441,w:1,m:0,tN:uWSGIWorker1Core2] User: wormball@gmail.com, LOCALDB: True
sqlalchemy.pool.impl.NullPool ERROR 2021-10-04 15:41:56,525 [pN:main.web.1,p:514441,w:1,m:0,tN:uWSGIWorker1Core1] Exception during reset or similar
Traceback (most recent call last):
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 697, in _finalize_fairy
    fairy._reset(pool)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 893, in _reset
    pool._dialect.do_rollback(self)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 558, in do_rollback
    dbapi_connection.rollback()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 140102974043904 and this is thread id 140102437172992.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 270, in _close_connection
    self._dialect.do_close(connection)
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 564, in do_close
    dbapi_connection.close()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 140102974043904 and this is thread id 140102437172992.
galaxy.tools.parameters.dynamic_options WARNING 2021-10-04 15:41:56,729 [pN:main.web.1,p:514441,w:1,m:0,tN:uWSGIWorker1Core1] could not filter by metadata: snpeff_db unknown
galaxy.tools.parameters.dynamic_options WARNING 2021-10-04 15:41:56,729 [pN:main.web.1,p:514441,w:1,m:0,tN:uWSGIWorker1Core1] could not filter by metadata: snpeff_db unknown
galaxy.tools.parameters.dynamic_options WARNING 2021-10-04 15:41:56,729 [pN:main.web.1,p:514441,w:1,m:0,tN:uWSGIWorker1Core1] could not filter by metadata: snpeff_db unknown
galaxy.tools.parameters.dynamic_options WARNING 2021-10-04 15:41:56,729 [pN:main.web.1,p:514441,w:1,m:0,tN:uWSGIWorker1Core1] could not filter by metadata: snpeff_db unknown
galaxy.tools.parameters.dynamic_options WARNING 2021-10-04 15:41:56,729 [pN:main.web.1,p:514441,w:1,m:0,tN:uWSGIWorker1Core1] could not filter by metadata: snpeff_db unknown
galaxy.tools.parameters.dynamic_options WARNING 2021-10-04 15:41:56,729 [pN:main.web.1,p:514441,w:1,m:0,tN:uWSGIWorker1Core1] could not filter by metadata: snpeff_db unknown
galaxy.tools.parameters.dynamic_options WARNING 2021-10-04 15:41:56,729 [pN:main.web.1,p:514441,w:1,m:0,tN:uWSGIWorker1Core1] could not filter by metadata: snpeff_db unknown
galaxy.jobs.handler ERROR 2021-10-04 15:42:08,088 [pN:main.web.1,p:514441,w:1,m:0,tN:JobHandlerStopQueue.monitor_thread] Exception in monitor_step
Traceback (most recent call last):
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/media/transgeneprep/DataStore/galaxy/.venv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
.....
.........
........

I tried several times but got about the same result (however number of scheduled steps was different every time). However scheduled steps are performed as charm (at least now that i corrected paths and invoked data managers). Similar behavior is also observed when i purge histories (but not when i delete history steps).

Is there a way to overcome these errors?

Thanks in advance.

Did you restarted galaxy (stopped it completely) between the retries and made sure there were no running processes anymore?

Several times.

Hmm strange, what if you excute the following command:
lsof <data_dir>/universe.sqlite the default data_dir is galaxy/database/.

If you get some results you could kill that process and try to start galaxy again.

transgeneprep@transgeneprep-System-Product-Name:/media/transgeneprep/DataStore/galaxy/database$ lsof universe.sqlite
lsof: WARNING: can't stat() fuse file system /root/.cache/doc
      Output information may be incomplete.
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /root/.cache/gvfs
      Output information may be incomplete.
transgeneprep@transgeneprep-System-Product-Name:/media/transgeneprep/DataStore/galaxy/database$ sudo lsof universe.sqlite
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
      Output information may be incomplete.
lsof: WARNING: can't stat() fuse file system /run/user/1000/doc
      Output information may be incomplete.

I tried again and got the same error. However now i also got this error running optitype:

docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/create: dial unix /var/run/docker.sock: connect: permission denied.
See 'docker run --help'.

But when i logged in and ran optitype again, it worked fine, however it logged me out again some time later without any apparent reason.

It is hard to debug. So in the basics a sqlite database can only be used by one process at the time. (Only one process can write to that database at the time and during writing it will be locked). There is a situation where it can be locked if the database file is located on a NFS-mounted or remote drive but if it worked before this is probably not the case. Here I read that it can also happen if you have unsaved actions Python SQLite: database is locked - Stack Overflow but I would not know how to solve that. Maybe if you dont have anything important in that database you could create a new one.

I installed galaxy again, and now all is working fine.

1 Like