Apache SSL Proxy Failing on 20.09

Apache 2 forwarding on CentOS 7 returns

"# Bad Request

Your browser sent a request that this server could not understand."

I get the same error whether I am running the browser on a remote client or on the server. I get the same error whether I use a file socket or port for proxying. When using a port I can connect directly to galaxy with no issues and access the galaxy website as expected.

When using file socket proxying I make sure both the galaxy and apache users have read and write access. The apache user is in the galaxy-admin group.

I am seeing SSL stapling warnings in the apache logs.

Below is the proxy config and the relevant galaxy.yml section. They are both taken directly from the galaxy 20.09 documentation with modifications for site paths and hostnames.

Any advice would be greatly appreciated:

There are a lot of commented lines but I wanted to include the whole section. I can edit to remove those.

uwsgi:
      # The address and port on which to listen.  By default, only listen to
      # localhost (galaxy will not be accessible over the network).  Use
      # ':8080' to listen on all available network interfaces.
      #http: 127.0.0.1:8888
      #https: 127.0.0.1:8443,/opt/galaxy/20.09/ssl/alliance-wildcard-public.crt,/opt/galaxy/20.09/ssl/alliance-wildcard.key
      #http-to-https: =0

      # listening options
      #shared-socket: :8888
      #shared-socket: :8443 #,ssl/alliance-wildcard-public.crt,ssl/alliance-wildcard.key
      #http: =0
      #https: =1
      #uid: galaxy
      #gid: galaxy-admin
      #static-map: /static=static

      # By default uWSGI allocates a very small buffer (4096 bytes) for the
      # headers of each request. If you start receiving "invalid request
      # block size" in your logs, it could mean you need a bigger buffer. We
      # recommend at least 16384.
      buffer-size: 16384

      # Number of web server (worker) processes to fork after the
      # application has loaded. If this is set to greater than 1,
      # thunder-lock likely should be enabled below.
      processes: 1

      # Number of threads for each web server process.
      threads: 4

      # Number of threads for serving static content and handling internal
      # routing requests.
      offload-threads: 2

      # Mapping to serve static content.
      static-map: /static=static

      # Mapping to serve the favicon.
      static-map: /favicon.ico=static/favicon.ico

      # Allow serving certain assets out of `client`.  Most modern Galaxy
      # interfaces bundle all of this, but some older pages still serve
      # these via symlink, requiring this rule.
      static-safe: client/src/assets

      # Enable the master process manager. Disabled by default for maximum
      # compatibility with CTRL+C, but should be enabled for use with
      # --daemon and/or production deployments.
      #master: false
      master: true

      # Path to the application's Python virtual environment. If using Conda
      # for Galaxy's framework dependencies (not tools!), do not set this.
      virtualenv: .venv

      # Path to the application's Python library.
      pythonpath: lib

      # The entry point which returns the web application (e.g. Galaxy,
      # Reports, etc.) that you are loading.
      module: galaxy.webapps.galaxy.buildapp:uwsgi_app()

      # Mount the web application (e.g. Galaxy, Reports, etc.) at the given
      # URL prefix. Cannot be used together with 'module:' above.
      #mount: /galaxy=galaxy.webapps.galaxy.buildapp:uwsgi_app()
      socket: /srv/galaxy/var/uwsgi.sock
      #mount: /galaxy=galaxy.webapps.galaxy.buildapp:uwsgi_app()
      #manage-script-name: true

      # Make uWSGI rewrite PATH_INFO and SCRIPT_NAME according to
      # mount-points. Set this to true if a URL prefix is used.
      manage-script-name: false

      # It is usually a good idea to set this to ``true`` if processes is
      # greater than 1.
      thunder-lock: false

      # Cause uWSGI to respect the traditional behavior of dying on SIGTERM
      # (its default is to brutally reload workers)
      die-on-term: true

      # Cause uWSGI to gracefully reload workers and mules upon receipt of
      # SIGINT (its default is to brutally kill workers)
      hook-master-start: unix_signal:2 gracefully_kill_them_all

      # Cause uWSGI to gracefully reload workers and mules upon receipt of
      # SIGTERM (its default is to brutally kill workers)
      hook-master-start: unix_signal:15 gracefully_kill_them_all

      # Feature necessary for proper mule signal handling
      py-call-osafterfork: true

      # Ensure application threads will run if `threads` is unset.
      enable-threads: true

      # uWSGI default umask. On some systems uWSGI has a default umask of
      # 000, for Galaxy a somewhat safer default is chosen. If Galaxy
      # submits jobs as real user then all users needs to be able to read
      # the files, i.e. the umask needs to be '022' or the Galaxy users need
      # to be in the same group as the Galaxy system user
      umask: 027

and the proxy config:

    SSLProtocol             all -SSLv3
    SSLCipherSuite          ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256
    SSLHonorCipherOrder     on
    SSLCompression          off
    SSLSessionTickets       off

    # OCSP stapling
    SSLUseStapling          on
    SSLStaplingResponderTimeout 5
    SSLStaplingReturnResponderErrors off
    SSLStaplingCache        shmcb:/var/run/ocsp(128000)

    #<VirtualHost _default_:80> 
    #    Redirect permanent / https://galaxy.example.org
    #</VirtualHost>

    <VirtualHost _default_:443>
        SSLEngine on
        SSLCertificateFile /etc/ssl/2020/public/alliance-wildcard-public.crt
        SSLCertificateKeyFile /etc/ssl/2018/private/alliance-wildcard.key

        # Enable HSTS
        Header always set Strict-Transport-Security "max-age=15552000; includeSubdomains"

        # use a variable for convenience
        Define galaxy_root /opt/galaxy/20.09

        # don't decode encoded slashes in path info
        AllowEncodedSlashes NoDecode

        # enable compression on all relevant types
        AddOutputFilterByType DEFLATE text/html text/plain text/xml
        AddOutputFilterByType DEFLATE text/css
        AddOutputFilterByType DEFLATE application/x-javascript application/javascript application/ecmascript
        AddOutputFilterByType DEFLATE application/rss+xml
        AddOutputFilterByType DEFLATE application/xml
        AddOutputFilterByType DEFLATE application/json

        # allow access to static content
        <Directory "${galaxy_root}/static">
            AllowOverride None
            Require all granted
        </Directory>

        # Galaxy needs to know that this is https for generating URLs
        RequestHeader set X-URL-SCHEME "%{REQUEST_SCHEME}e"

        # allow up to 3 minutes for Galaxy to respond to slow requests before timing out
        ProxyTimeout 180

        # proxy all requests not matching other locations to uWSGI
        ProxyPass / unix:///srv/galaxy/var/uwsgi.sock|uwsgi://
        # or uWSGI on a TCP socket
        #ProxyPass / uwsgi://127.0.0.1:4001/

        # serve framework static content
        RewriteEngine On
        RewriteRule ^/static/(.*) ${galaxy_root}/static/$1 [L]
        RewriteRule ^/favicon.ico ${galaxy_root}/static/favicon.ico [L]
        RewriteRule ^/robots.txt ${galaxy_root}/static/robots.txt [L]

        # enable caching on static content
        <Location "/static">
            ExpiresActive On
            ExpiresDefault "access plus 24 hours"
        </Location>

        # serve visualization and interactive environment plugin static content
        <Directory "${galaxy_root}/config/plugins/(.+)/(.+)/static">
            AllowOverride None
            Require all granted
        </Directory>
        RewriteRule ^/plugins/(.+)/(.+)/static/(.*)$ ${galaxy_root}/config/plugins/$1/$2/static/$3 [L]
    </VirtualHost>

The doc is wrong for Apache mod_proxy_uwsgi.
It should be:

ProxyPass / unix:///srv/galaxy/var/uwsgi.sock|uwsgi://localhost/

Question: how to notify Galaxy that some points in the Galaxy documentation are wrong or not documented enough

2 Likes

Yes, You are right.
I have faced the same issue with Apache proxy for Galaxy.
I have also updated my apache with the same things and its works in Galaxy v20.01 for the SubURL proxy path

ProxyPass /galaxy ProxyPass unix:///<my_path>/galaxy/uwsgi.sock|uwsgi://localhost/

Galaxy documentation must be updated for their errors as all Galaxy administrators are refereeing for their configuration without much in-depth details about it.
Thanks

I think you can open a pull request in this page

1 Like