upload_from_ftp is broken

usegalaxy.org support
,
1

Previously calling gi.tools.upload_from_ftp would return a dictonary like this (as stated in the documentation):

{'implicit_collections': [],
 'jobs': [{'create_time': '2015-07-28T17:57:43.704394',
           'exit_code': None,
           'id': '82b264d8c3d11790',
           'model_class': 'Job',
           'state': 'new',
           'tool_id': 'upload1',
           'update_time': '2015-07-28T17:57:43.910958'}],
 'output_collections': [],
 'outputs': [{'create_time': '2015-07-28T17:57:43.209041',
              'data_type': 'galaxy.datatypes.data.Text',
              'deleted': False,
              'file_ext': 'auto',
              'file_size': 0,
              'genome_build': '?',
              'hda_ldda': 'hda',
              'hid': 17,
              'history_content_type': 'dataset',
              'history_id': 'f3c2b0f3ecac9f02',
              'id': 'a676e8f07209a3be',
              'metadata_data_lines': None,
              'metadata_dbkey': '?',
              'misc_blurb': None,
              'misc_info': None,
              'model_class': 'HistoryDatasetAssociation',
              'name': 'test.txt',
              'output_name': 'output0',
              'peek': '<table cellspacing="0" cellpadding="3"></table>',
              'purged': False,
              'state': 'queued',
              'tags': [],
              'update_time': '2015-07-28T17:57:43.544407',
              'uuid': '2cbe8f0a-4019-47c4-87e2-005ce35b8449',
              'visible': True}]}

Now it seems after I call upload_from_ftp, I get a list of 60 dicts instead, and it seems to me that it’s returning the list of tools that you see on the left-hand column when you log-in to usegalaxy.org, but I have no idea why.

How can I have it return the dataset that was just uploaded? In fact, I don’t think the file was uploaded from FTP because I can’t see it in the history.

I’m using Python 3.6.9 and BioBlend 0.13.0.

2

This is still the dictionary being returned for me today, both locally and usegalaxy.org.

Can you make sure that

  • the URL is correct
  • the filename is correct (the FTP file needs to exist)
  • your API key is correct

?

3

Hi Marius,

I work with @Dorde_Popovic.

Here’s a distilled version of the code we use:

from bioblend import galaxy

API_KEY='REDACTED_KEY'

gi = galaxy.GalaxyInstance(url='usegalaxy.org', key=API_KEY)
gi.histories.get_most_recently_used_history()
hist_id = gi.histories.get_most_recently_used_history()["id"]
file_to_upload = gi.ftpfiles.get_ftp_files()
print(file_to_upload)
output = gi.tools.upload_from_ftp(file_to_upload[0]["path"], hist_id)
print(len(output))
for obj in output:
  if 'name' in obj:
    print(obj['name'])

Here’s the output of the above snippet:

[{'path': 'TESTUPLOAD.txt', 'size': 240, 'ctime': '02/25/2020 02:10:53 AM'}, {'path': 'test.bw', 'size': 27, 'ctime': '02/25/2020 02:10:43 AM'}]                                              
60
Get Data
Collection Operations
Expression Tools
Text Manipulation
Filter and Sort
Join, Subtract and Group
Datamash
FASTA/FASTQ
FASTQ Quality Control
SAM/BAM
BED
VCF/BCF
Nanopore
Convert Formats
Lift-Over
Operate on Genomic Intervals
Fetch Sequences/Alignments
Assembly
Annotation
Mapping
Variant Calling
ChIP-seq
RNA-seq
Multiple Alignments
Phenotype Association
Evolution
Regional Variation
STR-FM: Microsatellite Analysis
Chromosome Conformation
Metagenomic Analysis
Mothur
Picard
deepTools
EMBOSS
GATK
NCBI BLAST+
HyPhy
RSeQC
MiModD
Du Novo
Seqtk
Transposon Insertion Sequencing
Genome Diversity
IWTomics
pRESTO
PlantTribes
Motif Tools
Statistics
Graph/Display Data
CloudMap (deprecated)
RNA Analysis (deprecated)

Note that we’ve been using the original script for the past year without problems, but it gave us this output when we tried it a couple of weeks ago.

If needed I can privately message you our API key.

Thank you for your time.

4

The script looks fine. When did you last try this ? Is there any redirection or proxying on your local network ?

5

The script looks fine. When did you last try this ?

I just tried the script now with a regenerated API key and it’s still giving me the same output.

Is there any redirection or proxying on your local network ?

AFAIK, not really, and I’ve also tried this script on different networks as well as on 2 different machines. I’ve even tried using the API key from a colleague’s account and I see the same behavior.

Any idea on how to proceed?

6

Try with the full url, gi = galaxy.GalaxyInstance(url='https://usegalaxy.org', key=API_KEY).
There may be a redirect without the protocol, and POSTs to a redirect often (or always?) become GETs.

7

Trying the full url worked!

Thanks for the help! :slight_smile:

I had made a GitHub issue for this on the bioblend repo; I’ll link to this post and close it.