Moving Galaxy 19.09 from python2 to Python3

Mathew_Moore · December 13, 2019, 7:06pm

Hi Guys, I’m guessing this is probably for Marius
I just upgraded to 19.09, and I’d like to move from python2 to python3. Is there any current documentation on this… I’m a little hesitant to boldly remove and re-create a venv without asking if there are any potential catches. Standard install and galaxy.yml, job-handlers pushing jobs to a small slurm cluster.

Regards,
Mat

marten · December 13, 2019, 8:43pm

There shouldn’t be any hard gotchas. We have been running https://test.galaxyproject.org/ on Python3 for about a year.

Mathew_Moore · December 13, 2019, 9:03pm

Thanks, Marten,
I’ll take it down over the weekend and see how the update goes
Mat

blankenberg · December 13, 2019, 9:59pm

Should be pretty straightforward:

stop galaxy
backup your old .venv: $ mv .venv .venv.bac
create a new venv with the correct python:$ virtualenv -p python3 .venv
start galaxy back up

If anything goes wrong, just restore the .venv.bac

Mathew_Moore · December 13, 2019, 10:12pm

Thanks, that’s what I thought, but it’s nice to have a second opinion before blindly jumping into a void…

Mathew_Moore · December 15, 2019, 11:12pm

Ok, I spoke too soon!
I deleted the .venv, created a python3.6 venv, and restarted galaxy… I get the following error, and galaxy fails to start

Traceback (most recent call last):
  File "lib/galaxy/web/framework/middleware/error.py", line 154, in __call__
    app_iter = self.application(environ, sr_checker)
  File "/home/galaxy/galaxy_home/galaxy/.venv/lib/python3.6/site-packages/paste/recursive.py", line 85, in __call__
    return self.application(environ, start_response)
  File "/home/galaxy/galaxy_home/galaxy/.venv/lib/python3.6/site-packages/paste/httpexceptions.py", line 640, in __call__
    return self.application(environ, start_response)
  File "lib/galaxy/web/framework/base.py", line 143, in __call__
    return self.handle_request(environ, start_response)
  File "lib/galaxy/web/framework/base.py", line 222, in handle_request
    body = method(trans, **kwargs)
  File "lib/galaxy/webapps/galaxy/controllers/root.py", line 79, in index
    return self._bootstrapped_client(trans)
  File "lib/galaxy/webapps/base/controller.py", line 281, in _bootstrapped_client
    return self.template(trans, app_name, options=js_options, **kwd)
  File "lib/galaxy/webapps/base/controller.py", line 360, in template
    masthead=masthead
  File "lib/galaxy/web/framework/webapp.py", line 936, in fill_template
    return self.fill_template_mako(filename, **kwargs)
  File "lib/galaxy/web/framework/webapp.py", line 949, in fill_template_mako
    return template.render(**data)
  File "/home/galaxy/galaxy_home/galaxy/.venv/lib/python3.6/site-packages/mako/template.py", line 476, in render
    return runtime._render(self, self.callable_, args, data)
  File "/home/galaxy/galaxy_home/galaxy/.venv/lib/python3.6/site-packages/mako/runtime.py", line 883, in _render
    **_kwargs_for_callable(callable_, data)
  File "/home/galaxy/galaxy_home/galaxy/.venv/lib/python3.6/site-packages/mako/runtime.py", line 920, in _render_context
    _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
  File "/home/galaxy/galaxy_home/galaxy/.venv/lib/python3.6/site-packages/mako/runtime.py", line 947, in _exec_template
    callable_(context, *args, **kwargs)
  File "/home/galaxy/galaxy_home/galaxy/database/compiled_templates/js-app.mako.py", line 48, in render_body
    __M_writer(unicode( h.url_for( '/' ) ))
**NameError: name 'unicode' is not defined**

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "lib/galaxy/web/framework/middleware/batch.py", line 80, in __call__
    return self.application(environ, start_response)
  File "lib/galaxy/web/framework/middleware/request_id.py", line 15, in __call__
    return self.app(environ, start_response)
  File "lib/galaxy/web/framework/middleware/xforwardedhost.py", line 23, in __call__
    return self.app(environ, start_response)
  File "lib/galaxy/web/framework/middleware/translogger.py", line 71, in __call__
    return self.application(environ, replacement_start_response)
  File "lib/galaxy/web/framework/middleware/error.py", line 164, in __call__
    exc_info)
  File "lib/galaxy/web/framework/middleware/translogger.py", line 70, in replacement_start_response
    return start_response(status, headers, exc_info)
SystemError: <built-in function uwsgi_spit> returned a result with an error set
[pid: 2784|app: 0|req: 2/3] 192.168.34.79 () {74 vars in 1418 bytes} [Sun Dec 15 22:51:21 2019] GET / => generated 0 bytes in 43 msecs (HTTP/1.1 500) 1 headers in 63 bytes (0 switches on core 1)
192.168.34.79 - - [15/Dec/2019:22:52:02 +0000] "GET / HTTP/1.1" 500 - "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.79 Safari/537.36"
Traceback (most recent call last):
  File "lib/galaxy/web/framework/middleware/error.py", line 154, in __call__
    app_iter = self.application(environ, sr_checker)
  File "/home/galaxy/galaxy_home/galaxy/.venv/lib/python3.6/site-packages/paste/recursive.py", line 85, in __call__
    return self.application(environ, start_response)
  File "/home/galaxy/galaxy_home/galaxy/.venv/lib/python3.6/site-packages/paste/httpexceptions.py", line 640, in __call__
    return self.application(environ, start_response)
  File "lib/galaxy/web/framework/base.py", line 143, in __call__
    return self.handle_request(environ, start_response)
  File "lib/galaxy/web/framework/base.py", line 222, in handle_request
    body = method(trans, **kwargs)
  File "lib/galaxy/webapps/galaxy/controllers/root.py", line 79, in index
    return self._bootstrapped_client(trans)
  File "lib/galaxy/webapps/base/controller.py", line 281, in _bootstrapped_client
    return self.template(trans, app_name, options=js_options, **kwd)
  File "lib/galaxy/webapps/base/controller.py", line 360, in template
    masthead=masthead
  File "lib/galaxy/web/framework/webapp.py", line 936, in fill_template
    return self.fill_template_mako(filename, **kwargs)
  File "lib/galaxy/web/framework/webapp.py", line 949, in fill_template_mako
    return template.render(**data)
  File "/home/galaxy/galaxy_home/galaxy/.venv/lib/python3.6/site-packages/mako/template.py", line 476, in render
    return runtime._render(self, self.callable_, args, data)
  File "/home/galaxy/galaxy_home/galaxy/.venv/lib/python3.6/site-packages/mako/runtime.py", line 883, in _render
    **_kwargs_for_callable(callable_, data)
  File "/home/galaxy/galaxy_home/galaxy/.venv/lib/python3.6/site-packages/mako/runtime.py", line 920, in _render_context
    _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
  File "/home/galaxy/galaxy_home/galaxy/.venv/lib/python3.6/site-packages/mako/runtime.py", line 947, in _exec_template
    callable_(context, *args, **kwargs)
  File "/home/galaxy/galaxy_home/galaxy/database/compiled_templates/js-app.mako.py", line 48, in render_body
    __M_writer(unicode( h.url_for( '/' ) ))
**NameError: name 'unicode' is not defined**

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "lib/galaxy/web/framework/middleware/batch.py", line 80, in __call__
    return self.application(environ, start_response)
  File "lib/galaxy/web/framework/middleware/request_id.py", line 15, in __call__
    return self.app(environ, start_response)
  File "lib/galaxy/web/framework/middleware/xforwardedhost.py", line 23, in __call__
    return self.app(environ, start_response)
  File "lib/galaxy/web/framework/middleware/translogger.py", line 71, in __call__
    return self.application(environ, replacement_start_response)
  File "lib/galaxy/web/framework/middleware/error.py", line 164, in __call__
    exc_info)
  File "lib/galaxy/web/framework/middleware/translogger.py", line 70, in replacement_start_response
    return start_response(status, headers, exc_info)
SystemError: <built-in function uwsgi_spit> returned a result with an error set
[pid: 2784|app: 0|req: 3/4] 192.168.34.79 () {74 vars in 1418 bytes} [Sun Dec 15 22:52:02 2019] GET / => generated 0 bytes in 47 msecs (HTTP/1.1 500) 1 headers in 63 bytes (0 switches on core 2)

I assume this is some historical python 2 code that survived, as I understand unicode was renamed in python3 …

Has anyone else had this issue? and before I go looking through the code, does anyone a have a solution?

Cheers,

Mat

marten · December 16, 2019, 5:04pm

I suggest you wipe this folder. To be extra thorough you can remove all *.pyc files everywhere in galaxy folders.

Mathew_Moore · December 16, 2019, 5:11pm

Thanks,
I’ll probably I’ll wipe that folder and all *.pyc files with vengeance next weekend. Our instance runs 24/7 during the week.

For the record, I had no issues restoring the python2.7 .venv. I just copied the backup back to .venv

Regards,

Mat

Topic		Replies	Views
Galaxy local venv installation error galaxy-local , python3	4	959	February 8, 2022
Problems after upgrading Ubuntu to 22 galaxy-local	5	969	November 16, 2022
Unicode error after migrating to python3.5 usegalaxy.org support server-admin	1	742	December 9, 2019
Error when trying to run Galaxy on MacOs usegalaxy.org support server-admin , galaxy-local , galaxy_1901 , virtualenv	12	2690	August 19, 2019
Enter Galaxy .venv python shell (for debugging) usegalaxy.org support server-admin	1	210	March 2, 2023

Moving Galaxy 19.09 from python2 to Python3

Related topics