Problems downloading some collections from .eu

I’m still doing experiments to try and pin down the exact cause, but I’ve ended up with a history where I can’t download at least some dataset collections. Some of this is stored on Backblaze B2 and connected as Bring Your Own Storage, some of it not (but copied from the same BYOS):

  1. On Backblaze: https://usegalaxy.eu/api/dataset_collections/1a62e951797985fa/download
  2. Not on Backblaze: https://usegalaxy.eu/api/dataset_collections/565cf7ceaf9ff1ec/download

I am going to do some more experiments, but for either of these (TB-Profiler JSON output) the result for attempting to download the collection is a 503 error.

My next experiment is going to be to re-run the analysis workflow start-to-finish on .eu storage and see if the problem persists.

Hi @Peter_van_Heusden

Agree, this is probably BYOS related. Let’s ping @wm75 – can you help? Thanks!

I ran a smaller set of data through the same analysis workflow, history ID 136925a1d66d947a , and was able to download without problems. So I think this definitely seems to be a BYOS problem.

We have been debugging this, and it seems the issue can be fixed by [25.0] Make collection download asynchronous by davelopez · Pull Request #20561 · galaxyproject/galaxy · GitHub

Once it is merged, we will try to deploy the update shortly after.

Thanks for reporting and providing all the details!

3 Likes

The problem is now solved. We have applied new fixes and deployed them. Additionally, I have tested both the shared links and can confirm that they are now downloadable. However, since these are rather large collections, it takes a little longer to download.

2 Likes