The usegalaxy.org FTP service will be decommissioned on August 12, 2022

Due to a number of factors, the usegalaxy.org FTP service will be decommissioned on August 12, 2022. The service was first introduced in 2010 for uploading large data to Galaxy in a time when browser upload functionality was severely limited. Since that time, browser limitations on file sizes have been removed and protocols for reliable, resumable uploads have been developed and integrated into the Galaxy platform. Because of this, the need for the FTP service has greatly decreased. In addition, the proliferation of institutional firewalls and data security policies have blocked increasing numbers of people from even being able to access the FTP service.

One important use case served by the FTP service remains, however: uploading large data from remote systems (where one only has command-line access to data, not a web browser). Because moving data to a system with a web browser can be tedious and inefficient, we have developed a solution: The galaxy-upload command-line utility. This tool interacts directly with Galaxy’s web interface, meaning that it is firewall friendly. It supports resuming interrupted uploads and specifying upload parameters directly, so there is no need for the separate “import from FTP” step that is necessary when using the FTP service.

galaxy-upload is a Python package installable via pip:

$ pip install galaxy-upload

Once installed, you can upload from the command line to a Galaxy history like so:

$ galaxy-upload --url https://usegalaxy.org \
    --api-key 70ffeec0ffeea11e1eaccede1337loaf --history-name 'Run 2' \
    RUN2_F_001.fastq.gz RUN2_R_001.fastq.gz

Full usage details can be found in the documentation at https://galaxy-upload.readthedocs.org/

The galaxy-upload utility is usable today; please switch at your convenience before the 12th, and post any questions or concerns to Galaxy Help or the galaxy-upload issue tracker.

3 Likes

This is a terrible decision. don’t know why you guys removing a good useful feature from the galaxy server. I used to upload huge files through FTP itself. I wish you revert back to the FTP Feature.

The decision to decommission the FTP service was not in our hands, a new organizational policy at our host institution dictated that FTP servers would no longer be allowed, and we complied with that policy.

1 Like

There’s also no size limit anymore for uploading files via your browser, uploads are resumable etc, mitigating the need to have a FTP server running.

I have tried uploading files via browser, but the success rate of uploading via browser is very low (40%) and time consumption is more as the transfer rate is low. Please revert back the FTP services if possible.

It is not possible, sorry. There are unlimited retries for the upload and there is no logical reason that this should be slower than FTP, so I would be interested in knowing what exactly is failing for you.

1 Like