Error when copy pasting results from Galaxy Europe to Galaxy US

usegalaxy.org support
, , ,
#1

Hi
I have been running into issues when using Unicycler on https://usegalaxy.eu/ (job fails in seconds).
So I have been wanting to try to repeat the job on https://usegalaxy.org/ as sometimes something won’t work on a server but will on another one.
To do so I needed my long reads so I used the “copy link” to import my data to the US server.
But I get an error message after a few seconds:

error
An error occurred with this dataset:
The uploaded file contains invalid HTML content

What I tried to transfer is a fastq file originating from filtlong (ran on the European server with no error).
Can anybody help?

Thanks
Vanina

2 Likes
#2

Hi @vanina.guernier

Start by double-checking that the data is really in fastq format and contains sequences (only).

If not, a format/content issue could explain both problems.

FAQs that can help more: https://galaxyproject.org/support/#getting-inputs-right

#3

@vanina.guernier I’d say it depends on which link you copied.
The one that will work reliably is the one you get while hovering over the download icon.

The one you get from the eye icon, in contrast, will work for some files, but, e.g., not for large fastq files (cause it links to the preview of the files, which is the html-formatted page with the warning that you’re viewing a large file).

Hope that helps,
Wolfgang

1 Like
#4

Thanks @wm75 . I used the download icon and it worked well for other large data, just not this specific data.
Vanina

#5

I don’t see anything wrong with the data.
The format is fastqsanger which should be fine I think.

I managed to import the MinIon data and do the filtlong analysis again on the US server.
But the Unicycler analysis that I launched has been on hold for about 5 days now.
Any troubles with this tool at the moment? I know I can sometimes expect delays, but would like to make sure it’s normal.

The same analysis failed straight away on https://usegalaxy.eu
Screen Shot 2020-08-25 at 11.41.25 AM
Should I put a specific request for Unicycler error message on https://usegalaxy.eu? (and it’s not related to the copy pasting topic)
This is the first time I get the message in blue, not sure what this means.

Thanks
Vanina

#6

Have you tried this more than once? It is possible one of the servers was just busy. How large was the data? Is it in a collection or an individual dataset?

You can double-check that the format is actually fastqsanger by redetecting the datatype. Use the pencil icon to reach: Edit attributes > Datatype > use the button to “detect datatype”. If you do not get fastqsanger as the result there is a content or format problem that makes the data out-of-specification. Some of these tools also process fasta data – and if the data is in that format, the correct datatype should be assigned.

Yes, the queue is quite long for jobs that run on the cluster that processes Unicyler. The same is true for Trinity and RNA-Star. Confirm that the inputs are Ok, and if they are, then allow any queued jobs to stay queued to ensure the fastest processing. If you delete and rerun, that will place the new jobs back at the end of the queue again, extending wait time. If the inputs are problematic – fix as needed, then go ahead and rerun.

Confirming your inputs are Ok can help here too. Also, make sure you are using the most current version of the tool (and all tools). A job failing so quickly is a strong indicator of an input problem. If the inputs are Ok, then you can submit a bug report. Putting some details into the comments would help – including a link to this Galaxy Help topic for context. Or @wm75 may ask for something different (he is an admin for the EU server).

The usegalaxy.eu server automatically reruns jobs that fail – and that is what the message/blue box is notifying you about. This helps to eliminate transient cluster/server-side issues (rare, but happen!). A rerun for any odd failure is a good idea. Not all public Galaxy servers do this automatically – but you can rerun yourself.

1 Like
#7

Yes, @vanina.guernier, please report that separate issue using the bug icon and submitting an error report to us from the usegalaxy.eu team. Maybe it also helps us diagnose what’s wrong with the data transfer, in which case I would post the findings back here.

#8

Hi. I reported both problems:

  • failed importation of MinIon data in fastqsanger (original data imported and filtered with filtlong on the EU server)
  • failed Unicycler jobs on the EU server
    I tried to run the same Unicycler job on the US server see if the error is consistent but still queuing.
    Thanks for help!
2 Likes
#9

Thanks for sending in the bug report. Using default server-settings at usegalaxy.eu required that the history and the datasets inside it were set to a “shared by link” state in order to transfer a dataset from usegalaxy.eu to usegalaxy.org. @wm75 may have comments about whether that is expected or not.

To be clear – you don’t need to publish the history/data – just set the history/datasets as shared by link. Only you or anyone you give that share link to would have access to your data (or server administrators). Do this under the History menu (gear icon) > choose “Share or Publish” > and on that form click the button to share by URL. Be sure to check the box to “also share datasets”. Once data is transferred, you can unshare as wanted.

More about data privacy, and how you can configure your own data permissions at any Galaxy server (assuming it is running a current release, as both of these public servers are), plus how to share Galaxy objects are all covered in these FAQs. You have full control over permissions and sharing. Galaxy Support - Galaxy Community Hub

As an aside, I didn’t notice anything obviously problematic with your data formatting or content – this is definitely long read fastqsanger. But the EU team can provide more feedback based on your bug report to them regarding the failed Unicyler job there. The job at ORG is still queued.

Hope that helps!

1 Like
#10

That’s a very good observation @jennaj (learnt something about our server config from it ;))

Whether or not accessing (and thus uploading to a different server) datasets via known links works or not by default is controlled by your settings in User -> Preferences -> Set Dataset Permissions for New Histories. In there you have an access category, and users that you have listed under it will have link-based access to any of your datasets. Now if you leave this list empty, it means any user has access.

Apparently, the default setting for newly created accounts has changed on usegalaxy.eu (maybe also on usegalaxy.org?) at some point (about 2 years ago it seems). Old accounts (like mine) have an empty list, while newer accounts start out with just the individual user’s email.
If you have an account with restricted access, you will have to Share via Link to make data tarnsfers work as @jennai pointed out. If you have the unrestricted setting, data transfers will work out of the box (at the cost of your datasets being potentially discoverable by anyone).

Also note that the setting is called “Set Dataset Permissions for New Histories”. Whatever changes you make to it will, thus, not change properties of datasets in existing histories!

1 Like
#11

Thanks! I couldn’t figure out the root difference, this helps a great deal!

Old and brand new accounts at usegalaxy.org do not have any users listed under User -> Preferences -> Set Dataset Permissions for New Histories > “access”.

We could consider updating the privacy FAQ to describe the defaults for all usegalaxy.* servers. When the enhanced data privacy settings were added to Galaxy (circa 2017), the FAQ was created. Currently, usegalaxy.org has the default policy stated and others are not clarified directly. Could add more content: directly stated or with a link to a server-specific policy maintained elsewhere. Whichever is easier to keep current. Let me know if I can help. cc @Slugger70

https://galaxyproject.org/learn/privacy-features/#how-is-the-server-im-working-at-configured

@vanina.guernier thank you all the help around troubleshooting this!