Fastq files not uploading

Fastq files have not uploaded for the past couple of hours, but no outage indicated on the status page.

Hi @imperialadmiral121

How are you loading data up to the UseGalaxy.eu server? FTP, URL, local file?

Let’s start there, thanks! :slight_smile:

XRef → Getting Data into Galaxy

Hi, I am uploading local files

It looks like some of the jobs that were stuck processing have started completing just now. I will monitor and update

Ok, great! :rocket:

Yes, I ran a test and it completed quickly. Very small files but the queue should work the same for large or small.

In general how the job queue works is below. Everyone has some set of concurrent “jobs” allowed. This means for a larger batch of data loading or tool runs, some may run and complete before the others. Leaving items queued is the important part.

I am able to run already-uploaded files, but the processing of uploaded files seems to have stopped working for me (even with a small test file uploaded using a different device remains gray). I will check again tomorrow

Update: no progress overnight. I tried uploading the small test file on the usegalaxy.org server, and it was slower than normal but completed after ~5min.

Hi @imperialadmiral121

I’m glad you were able to load to the UseGalaxy.org server as a work-around!

The UseGalaxy.eu server just seems to be busy. I see many unscheduled and queued jobs for the Upload tool – named DATA FETCH in these graphs.

Where → How to see the UseGalaxy.eu job queue statistics


The best advice we can give is to get your data queued then allow it to process. For local data, you’ll need to stay connected. For cloud data, this will process while you are away. The same is true when it comes time to download the results, too.

Hope this helps! The EU administrators are watching what we are talking about here and will jump in if there is anything special to report. :slight_smile:

Thanks for the info!

1 Like

Update: usegalaxy.eu server uploads still not processing for me. Tried creating new histories, and switching between short-term and long-term storage. usegalaxy.org is working fine.

Update: working again. One file did not upload correctly due to “bus error (core dumped),” but everything else is processing.

1 Like

Thanks for the updates @imperialadmiral121 and I’m glad the ORG server is working for you.

Let’s ping the EU admins. Maybe they can provide more context. @wm75 – is there anything special going on or do you have any advice to add? Thanks!

Well, we definitely had recurring upload issues on Galaxy Europe since the weekend.

Things are currently working and we hope it stays that way now.

Sorry for the inconvenience this has caused you,

Wolfgang

1 Like

I seem to be having the same kind of issue with usegalaxy.eu today. usegalaxy.org is working fine

The files that were in progress (orange) have still not resolved, but the files that were still waiting to run (grey) have now completed (including the in-progress files that I re-queued).

Hi @imperialadmiral121

How are your Upload jobs now?

The UseGalaxy.org server had some maintenance downtime in early January, but not last week. If you have some data that has been queued for “too long” at UseGalaxy.org I would be interested in looking at it. Shared history link? Screenshot? If you include your username and history name in the screenshot, I can take a peek into the other details directly and provide some feedback.

Please let us know! Thanks! :slight_smile:

The usegalaxy.org server was working fine; I was having issues with the usegalaxy.eu server. I believe that I ended up deleting the in-progress files that had not resolved (as the re-queued files had already completed). Thanks for following up

Hello !

First of all, sorry for the issues caused using EU instance. We are currently facing some problems regarding our htcondor runner. We are not quiet sure yet what the root cause is and how we can fixed.

Some of the jobs could indeed stay in orange (queued) state and do not finish. This is quiet random and we cannot predict witch jobs will stay in this “state”. Internally, the condor queue finished the job, but the handler doesn’t pick up on it. For now we can force a restart on handlers

We are working on a solution and hope to fix this soon as possible. Thanks for reporting the issue!

Gabriel

2 Likes

Thanks for the explanation!

This should be mostly fixed since some days ago. Hope your jobs are not staying orange anymore :slight_smile:

1 Like