Normally not a problem, though lately been extremely slow. Now all I get is an error when I try to upload to galaxy.org I tried copying link and pasting to galaxy.org, instant error. Tried copying history url and uploading via the new Beta upload, same issue.
Hi @Jon_Colman,
It seems the ORG server has an issue with incorrect quota calculation affecting data upload. Check Disc quota exceeded topic for more info. I guess we have to wait until it is resolved.
Kind regards,
Igor
I just noticed another issue, I was moving some files from .EU to Australia. File size 260.9MB, when it arrived as a completed upload it is now 14.8MB. I had a previous issue that was similar, moving a file from .EU to .ORG, pretty much the same thing. I will share the folders
Galaxy File 609, the second file currently in process is 606
Galaxy | Australia Uploaded to Australia as File 2 Unknown_R2
Hi @Jon_Colman
Galaxy Europe is currently inaccessible. I’ll check the original files later, if needed.
I see two files in the shared history on Galaxy Australia, R1 is 250 MB, and R2 ~14 MB. It seems R2 file is damaged: FastQC failed with:
uk.ac.babraham.FastQC.Sequence.SequenceFormatException: Ran out of data in the middle of a fastq entry. Your file is probably truncated
It can happen occasionally, but, personally, it never happen to me for Galaxy-to-Galaxy transfer. Check if the original file is intact, for example, use FastQC or check the last four lines (you should see intact reads, with the quality string matching the sequence string). If the original file is intact, repeat the transfer. Obviously, we need to wait until Galaxy Europe is back online.
Kind regards,
Igor
Ping @wm75 - Galaxy Europe is not accessible atm.
The original files are fine. It’s happened numerous times for me. One time I was moving about 5 data sets from Galaxy Australia to .EU, they all had problems that were similar. I can’t recall if they all happened on the newest site update or not. I have noticed a lot more freezing up on .EU recently. Things get slow in processing tasks, then things stop and it gives an error message, then reloads the data again and things are fine again. It’s fairly frequent.
Hi @Jon_Colman,
Thank you for the additional information! I usually transfer smaller files between Galaxies and have not noticed any issue. Disruption of data transfer may occur at start point, during transfer and at the destination server. There were some recent complains about data transfer and stability of Galaxy Europe, and it was coincided with the Galaxy conference in Europe. The conference is over, people are back to work. I hope the issue will be addressed.
Kind regards,
Igor
Hi @Jon_Colman,
I have impression that Galaxy Europe has stronger privacy settings compared to other servers, and it makes outbound data transfer using URL somewhat complicated. I tested data transfer for big datasets from Europe to Australia and got truncated files and “forbidden” errors for some transfer jobs. Here, big means between 250 Mb and 5 Gb. The files were mix of FASTQ and GZipped FASTQ. Making history accessible helped with truncated files issue, and “forbidden” errors were resolved by disabling “Make new datasets private”.
Hope that helps.
Kind regards,
Igor
Hi Igor, thanks. Its probably some new system settings.
Is there a way to send you a private message, I have some different questions I want to ask.
I think it looks like most likely because I was trying to move too many data sets at one time. When doing just one at a time it seems to be ok
I am glad the data transfer works for you now.
On the forum, new messages have an option for personal message: in the top left corner change Create topic to Create message.
Kind regards,
Igor
Ok, thanks!!!
Still having issues with the paired reads issue. So my Illumina files work perfectly, but not the Element Aviti, could it be something in the header??? Here are the 2 that I’m working with, could the front of the Aviti header possibly be causing issues???
Here is the Illumina
@LH00213:51:22FY3FLT3:2:1101:1425:1032 1:N:0:AACGTGAAGG+CTAATAACCG
Here is the Element Aviti
@AmpSeq-MD2:AVITI-Yicheng-Sensei-Jon-101224:2413435223:1:10102:0234:0025 1:N:0:CTCACCAAAT+TCACGCCGAC
Hi @Jon_Colman,
It seems you accidentally posted message into another topic. Considering the situation (successful processing of illumina data, failure for Aviti), it is possible that read names might be an issue. Check reverse reads, start of comment in the read name string. I expect illumina has 2:. What do you see in Aviti? By any chance, is it 3:? Check how people deal with not properly paired Aviti reads.
Please, do not mix different issues in one topic.
Kind regards,
Igor
I thought I kept them separate.
Anyways Aviti Read2 2:N:0
If I’m looking at things properly, though I don’t know what I “can’t” see in the BAM file, but technically the only difference is “AmpSeq-MD2:AVITI-Yicheng-Sensei-Jon-”, I haven’t seen anything on Aviti on this. Actually, it would be ideal if I could replace that portion of the header with “JMC09241943”
Is there a way to do this??? This would make he header in the same format as the Illumina.
I think I found a potential issue on the Aviti. I read that BAM format is Phred 33 mandatory, which I think ends at 41, most of Illumina is JJJJJ which I think is 40. With Aviti most of the scoring is NNNN or MMMM. I’m going to try again by running BBduk with no options, which I found converts to Phred 33. This may be causing errors with the BAM files.