Unable to curl download files from usegalaxy.eu

usegalaxy.eu support
,
1

I’ve been trying to download files from my history in usegalaxy.eu using ‘curl’, following the instructions on the following page: Downloading Data - Galaxy Community Hub

This method works fine with the main Galaxy server at usegalaxy.org, just not with the European server. I’m able to download files in a web browser using http and https.

As an example, using the verbose parameter, I get the following output (precise URLs redacted). Any advice would be much appreciated. Downloading ginormous BAM files through a web browser is not ideal.

info-sml-113-x:SSD bramblepuss$ curl -JLOv https://usegalaxy.eu/datasets/redacted/display?to_ext=bam
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:-- 0* Trying 192.52.32.126…

  • TCP_NODELAY set
  • Connected to usegalaxy.eu (192.52.32.126) port 443 (#0)
  • ALPN, offering h2
  • ALPN, offering http/1.1
  • Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
  • successfully set certificate verify locations:
  • CAfile: /etc/ssl/cert.pem
    CApath: none
  • TLSv1.2 (OUT), TLS handshake, Client hello (1):
    } [512 bytes data]
  • TLSv1.2 (IN), TLS handshake, Server hello (2):
    { [93 bytes data]
  • TLSv1.2 (IN), TLS handshake, Certificate (11):
    { [2636 bytes data]
  • TLSv1.2 (IN), TLS handshake, Server key exchange (12):
    { [333 bytes data]
  • TLSv1.2 (IN), TLS handshake, Server finished (14):
    { [4 bytes data]
  • TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
    } [70 bytes data]
  • TLSv1.2 (OUT), TLS change cipher, Client hello (1):
    } [1 bytes data]
  • TLSv1.2 (OUT), TLS handshake, Finished (20):
    } [16 bytes data]
  • TLSv1.2 (IN), TLS change cipher, Client hello (1):
    { [1 bytes data]
  • TLSv1.2 (IN), TLS handshake, Finished (20):
    { [16 bytes data]
  • SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
  • ALPN, server did not agree to a protocol
  • Server certificate:
  • subject: CN=usegalaxy.eu
  • start date: Dec 7 07:26:49 2018 GMT
  • expire date: Mar 7 07:26:49 2019 GMT
  • subjectAltName: host “usegalaxy.eu” matched cert’s “usegalaxy.eu”
  • issuer: C=US; O=Let’s Encrypt; CN=Let’s Encrypt Authority X3
  • SSL certificate verify ok.

GET /datasets/8fb8030d304f7807/display?to_ext=bam HTTP/1.1
Host: usegalaxy.eu
User-Agent: curl/7.54.0
Accept: /

< HTTP/1.1 200 OK
< Server: nginx/1.12.2
< Date: Wed, 16 Jan 2019 20:16:17 GMT
< Content-Type: text/html
< Transfer-Encoding: chunked
< Vary: Accept-Encoding
< Set-Cookie: galaxysession=11ac94870d0bb33aef1bd67c23e76ec7d7d0aa26dbc781503fab4ecc9b5ac968cb576e82072afe8f; expires=Tue, 16-Apr-2019 22:16:17 GMT; httponly; Max-Age=7776000; Path=/; secure; Version=1
< X-Content-Type-Options: nosniff
<
{ [4979 bytes data]
100 4960 0 4960 0 0 9185 0 --:–:-- --:–:-- --:–:-- 9202

1 Like
2

Hi - Just ran a few tests and it appears that the history and objects need to be shared before curl can be used for download at the EU server. This can just be a share link (no need to “publish” as well). However, it is important to check the box that “also shares objects”. If the history is already shared, unshare, check the box, then reshare.

Screenshot after clicking object share box but before clicking on the share-by-link:

share-history-objects
share-history-objects.png1284×272 33.7 KB

Here is an example dataset I’ll leave up for a bit so you can try it: tabular dataset, one line, super small

$ curl -o outfile_after_sharing 'https://usegalaxy.eu/datasets/531ddf9f9edb68b8/display?to_ext=txt'
$ more outfile_after_sharing
hello   world

I’ll follow-up with the EU team about this and get these details added to the general Download FAQ if “sharing first” is an intentional pre-requirement for that server (seems so from testing).

Hope that helps!

2 Likes