Hi @richard64
General instructions for Upload are in this guide → Getting Data into Galaxy
FTP is not available at all public Galaxy servers, since the Upload tool itself supports a resume function. The data loading is processing through the API, and works about the same way as FTP.
For batch loading of data, you can access the API directly on the command line. See the guide above for the details. The speed will be the same but this is convenient when there are many files, or when data loading, workflow execution, and result downloads are all run in batches.
What you can try:
-
Move to a faster internet connection. The rate of data loading is based on your connection bandwidth and the server bandwidth. The large public servers have a lot of capacity, so the limit is usually with the “from” connection.
-
Load the data to your private cloud data storage location first, then transfer the data to Galaxy.
This means getting the data up to servers just once, and you might want to export your results that way too. Server-to-server transfers are much faster since they don’t involve home internet connections that are optimized for “download” speeds, not “upload speeds”.
The latest release supports even more cloud providers, so look under User → Preferences for the options available at the server you are working at. Please ask if you need help with this.
-
Start the transfer in the background, and let it process.
You’ll need to keep your computer on throughout. You will be able to resume a broken connection for a short window, but will need to be responsive.
For your files, yes these might take some time. You could investigate your connection speed with a speedtest tool to see what it is at, then consider moving to faster connection. Not running other data transfers at the same time might help too (avoiding streaming, etc).
And, if this is something you plan to repeat, putting the data in Dropbox or one of the other options available to you is another way around this. Any public URL will work, so a local server that can host the data can work too. Working with large data from a personal computer over a home connection will always be the slowest option for getting data up to any server then back down again – doing this just once has strong advantages, even if less convenient.
Finally, if the data is from some public data source, transferring data directly from that source into Galaxy is the “best” way. Please ask if you need help with how.
If you want to share the URL of the server you are using, I can try a test, too, to double check that more is not going on with an independent test. Also please confirm, these are fastq.gz files, yes?
Let’s start there! 