Cannot get QIIME2 artifact to work

[mbyec18]

Hi all, I’m very new to galaxy and the associated pipelines used within this site first off! To sum up what I’m aiming to do in short is analyse microbiome data generated by NGS.

So far I’ve managed to run BLASTS within galaxy, trim my FASTQ data and do the basics such as quality control stats. However. I would like to go onto using Mothur or DADA2 to generate OTU’s but this is seemingly impossible!

So, I have tried to use the QIIME2 import pipeline within galaxy but every time I try I get an error message and red box around my job in the queue which looks like this: Dataset Error Report
An error occurred while running the tool toolshed.g2.bx.psu.edu/repos/q2d2/qiime2_core__tools__import/qiime2_core__tools__import/2024.5.0+dist.he540b0b0.
Details
Execution resulted in the following messages:

            Fatal error: Exit code 1 ()

Tool generated the following standard error:

        Unexpected error importing data: /cor

ral4/main/objects/a/8/0/dataset_a80a8
aad-e0be-4cfb-b3eb-4e3dca357791.dat
is not a(n) AlignedDNAFASTAFormat
file:
First line of file is not a valid
description. Descriptions must start
with ‘>’

Matplotlib created a temporary cache directory at /corral4/main/jobs/061/523/61523435/tmp/matplotlib-bqoej_fd because the default path (/home/qiime2/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
Traceback (most recent call last):
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/plugin/model/file_format.py”, line 26, in validate
self.validate(level)
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2_types/_util.py”, line 37, in validate
self._validate_FASTA(level, FASTAValidator, ValidationSet)
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2_types/_util.py”, line 70, in _validate_FASTA
raise ValidationError("First line of file is not a valid "
qiime2.core.exceptions.ValidationError: First line of file is not a valid description. Descriptions must start with ‘>’

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File “/opt/conda/envs/qiime2-amplicon-2024.5/bin/q2galaxy”, line 11, in
sys.exit(root())
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/click/core.py”, line 1157, in call
return self.main(*args, **kwargs)
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/click/core.py”, line 1078, in main
rv = self.invoke(ctx)
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/click/core.py”, line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/click/core.py”, line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/click/core.py”, line 783, in invoke
return callback(*args, **kwargs)
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2galaxy/main.py”, line 96, in run
builtin_runner(action, config)
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2galaxy/core/drivers/builtins.py”, line 41, in builtin_runner
tool(inputs, stdio=stdio)
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2galaxy/core/drivers/builtins.py”, line 62, in import_data
artifact = import_name_data(type, format, files_to_move,
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2galaxy/core/drivers/stdio.py”, line 38, in wrapped
return function(*args, **kwargs)
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2galaxy/core/drivers/builtins.py”, line 162, in import_name_data
return qiime2.Artifact.import_data(type, path, view_type=format)
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/sdk/result.py”, line 332, in import_data
return cls.from_view(type, view, view_type, provenance_capture,
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/sdk/result.py”, line 360, in _from_view
result = transformation(view, validate_level)
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/core/transform.py”, line 68, in transformation
self.validate(view, level=validate_level)
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/core/transform.py”, line 143, in validate
view.validate(level)
File “/opt/conda/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/plugin/model/file_format.py”, line 28, in validate
raise ValidationError(
qiime2.core.exceptions.ValidationError: /corral4/main/objects/a/8/0/dataset_a80a8aad-e0be-4cfb-b3eb-4e3dca357791.dat is not a(n) AlignedDNAFASTAFormat file:

First line of file is not a valid description. Descriptions must start with ‘>’

Not quite sure where I’m going wrong here, I know that I’m being told to use a file in ‘gza’ format! This means very little to me and I’m not sure how I would go about getting my files into this format. Finally, please be aware I have very minimal coding knowledge and would be hesitant to use the likes of Conda!

Thanks in advance.

[jennaj]

Welcome, @mbyec18

The first step with the Qiime tools is to get your data into an artifact. There are a few types, and there isn’t a lot of Galaxy specific documentation for this – just use the original documentation from the tool authors. Your current errors about finding the “wrong” artifact type.

Then the next step, is about if you are having trouble (this is where you are now), is to loop back to examine your artifact type/content against the type of artifact that the downstream tools is expecting to work with.

There is a web site the authors host where you can review the current artifact content, and their documentation explains what you will need instead. Then you’ll need to create that and try again. This can be a bit technical so start at the very first tutorial the authors host, and maybe visit their forum, since they know this software best. All qza artifacts are like a little database: files and metadata. What kind of data, and how it is organized, can vary a great deal!

We do have some prior QA here. One is where I walked through and created an artifact for someone, and the others have related discussions. Search results for 'qiime order:latest' - Galaxy Community Help

So, in short: get the right artifact created for your target tool done first to avoid this type of error. Then you can focus on that tool’s usage in more detail (data, parameters). Right now all of your errors are technical message about the formats (and not even the data format yet, instead the artifact format).

Or, you can back up and determine why the same tools, used outside of the Qiime package, are failing and address that instead. These will be regular files, and not nested into their own “database” the same way the artifacts are, so are a bit easier to work with, and more transparently.

Both Mothur and DADA2 have tutorials linked on the bottom of the tool forms. And if you have a specific error with those, we are much more likely to be able to help.

Thanks, and hope this helps to understand what is going on!

[mbyec18]

Hi @jennaj thank you so much for your response!

I think I already tried to convert my FASTq files using the ‘import to QIIME2’ function in galaxy but this doesn’t work for me either. How would I go about finding which kind artifact my data is currently in? And then once I know that, how can I change it?

I don’t think I actually have any metadata, I literally have two species of spiders that I am working on but the microbiome was sequenced from the whole body.

There are no additional details other than species name and a number to accompany the individual. Each sample has two outputs which I assume is the forward and reverse reads for each spider.

Sorry to ask any silly questions, this is my first go at this and the likes of code and manual commands terrifies me, so hopefully I can do what I need to with galaxy successfully. Thank you.

[jennaj]

Hi @mbyec18

Ok, for your case I would skip Qiime entirely.

Just use the regular tools in the tool panel. We have tutorials for these – did you check the bottom of the Mother and DADA2 tool forms? Those will link out to where you can start.

Or, go to the training site top level https://training.galaxyproject.org/ and go into your topic and review directly. Sometimes starting with the introduction tutorials helps anyway, since it explains what is actually going on from a scientific prospective, to help when things get more complicated technically later on. And, we can always help with any errors you are not sure about here.

If you want to dive into Qiime, you can try their very introduction tutorials later on the command line, use their support resources, then try those in Galaxy, too as a learning exercise. It is much more than we can cover at this forum, and you won’t really need them. They are a connivance feature in Galaxy, mostly for people used to using them other places who want to use our workflow engine.

[mbyec18]

Do you happen to have the link to the post where you did a walk through of the creating an artefact, please? Many thanks. @jennaj

[jennaj]

This is one of them → qiime2 tools import

Others can be found under qiime2_core__tools__import and qiime2

This tool does not have a Galaxy tutorial. Everything is in the original tool documentation, then on the Galaxy forms. Some is up in the form where the options are made and some is down in the Help section.