Hello,
Ive been trying to download a large collection ~32GB of alignments, and despite attempts, ive not been able to download more than a GB before it cuts out. this includes browser and wget options.
Any advice?
I should have mentioned, this is the public usegalaxy.org server, not a local one
Hello @Michael_Ridley
For large datasets, dataset collections and/or slower connections, use either curl or wget to download data. The exact commands are in the FAQ below.
If using MAC OSX, use the Terminal utility to get command-line access. If using a different OS, you can google around for the options.
FAQ: https://galaxyproject.org/support/
Hope that helps!
Hello Jennaj,
Even using wget/curl it cuts out after almost exactly 1Gb, is there another command to specify for it keep going, or is this likely to be a network issue?
1 Like
Yes, this could be a network issue. A slower connection can drop. Your internet provider could have download limits. Other connection issues are possible but those are two most common reasons we have had reported when this has come up before.
Try adding this argument to your wget string: -c
For more explanations, Google “resume curl” or “resume wget” for much discussion. Plus, the man page for each tool will have official argument descriptions. Both can resume but in slightly different ways that can also differ by OS. You’ll need to test what works for you. If none work, confirming what your IP provider data transfer/time limits are is the next step.
Example, this Q&A gives a good line-liner to "resume transfer for curl".