Hello!
I’m one of the lead developers of QIIME 2 and generally the developer behind q2galaxy which renders all of our actions into galaxy tools.
I was in the process of rerendering our latest release for publication on the toolshed, and ran into some lint errors from Planemo.
Since the features I’m using are deprecated, it doesn’t really seem like something for planemo’s issue tracker, since I’m already kind of doing the “wrong” thing.
Here’s some more context:
In QIIME 2 we abstract data into what we call artifacts which contain both a semantic type (which defines what kinds of actions you can run) and a format (which defines what kinds of automated transformations need to be done). This information, provenance, and the raw data itself (in some arbitrary but ideally obvious file format) are all zipped together into a single file we call a .qza (the artifact).
At the user interface level, the only relevant information is the semantic type, however this set is extensible and depends on the plugins that are defined. So to make QIIME 2 work in Galaxy, we defined a generic file format (QZA) which sets Galaxy metadata including the semantic type.
Then we can filter our allowable inputs based on the metadata, instead of the format. This gives us the ability to continue to extend the vocabulary of types, without needing to add a new format to Galaxy each time a plugin decides to do so (there are many, so this would be pretty intractable).
Currently, this is done with the following (deprecated, but suggested at BOSC 2018) syntax:
The weird part is the options_filter_attribute combined with filter add_value to create a set that is matched against the available QZAs. Again, this essentially is mimicing the format filtering that Galaxy does, but the at the metadata level. This however is a pretty obscure use, and I had to add support for multiple add_value flags to Galaxy at the time.
Then I added a validator for situations where the user may manually select some option not in the generated options list.
The issue I’m now encountering is that Planemo doesn’t recognize this as a valid combination:
+Linting tool /home/runner/work/galaxy-tools/galaxy-tools/tools/suite_qiime2__alignment/qiime2__alignment__mafft_add.xml
Failed linting
Applying linter tests... WARNING
.. WARNING: No tests found, most tools should define test cases.
Applying linter output... CHECK
.. INFO: 1 outputs found.
Applying linter inputs... FAIL
.. ERROR: Data parameter [alignment] filter needs to define a ref attribute
.. ERROR: Data parameter [alignment] for filters only type="data_meta" is allowed, found type="add_value"
.. ERROR: Data parameter [sequences] filter needs to define a ref attribute
.. ERROR: Data parameter [sequences] for filters only type="data_meta" is allowed, found type="add_value"
.. INFO: Found 7 input parameters.
So my question is, what should we do now? It seems like data_meta is almost, but not quite the kind of filter we need, as it seems to be concerned with the metadata of a sibling parameter. And there’s not a clear way to define a list of valid options.
What would be our best approach here? Is this actually an issue with Planemo, or have we just kind of fallen into the inevitable state of a deprecated feature.
I think it’s fair to say that there’s a substantial amount of interest in Galaxy from our users, so if there was a need to create an alternative non-deprecated way of making this happen (some kind of explicit metadata based filter), we could contribute towards that effort. In fact, our semantic types are actually a little bit more general than a list (they form an algebra amongst themselves), so improving support beyond that (I’m not sure how exactly at this moment) could actually be pretty helpful.
I do also acknowledge that the way QIIME 2 thinks about data is pretty alien relative to the world of workflow-description languages which generally have a pretty shallow concept of file formats. And that integrating with those may be the more relevant direction for your project, so we’re definitely coming from a completely unrelated angle which may not be of much interest.
But I am exceptionally pleased at how robust Galaxy’s interface has been, that filtering to the semantically relevant actions was possible at all. WDL and CWL (and interfaces for composing them) do not share that capacity, and make for pretty disappointing interface targets for us, relative to Galaxy’s tool XML.
Getting this working smoothly would be very good for everyone!
… Maybe I was wrong. If I was right I would assume that the new functionality added in this PR should be able to solve your task. Maybe worth a try.