Local Galaxy installation: Tool Shed dependencies as packages are not installed --Older tool without dependency resolvers incorporated

Dear Galaxy community,

I am quite new to Galaxy and have set up my own local Galaxy server for learning.
This is installed on Centos, configured with separate PostgreSQL DB and Apache as proxy.
So far the server worked well.

However, I have a problem when installing tools from the Tool Shed. More specifically with those tools that have dependencies that are not installed via Conda but via Package.
When I install such a tool (e.g. fml_gff3togtf) the installer tries to install the dependencies (here e.g. atlas or biopython) but fails. Under “Manage Tools” those dependencies are then listed as “Installed, missing tool dependencies”:

I had a look at the folders in “Database” and realized that the dependency package is downloaded in the temp folder, the dependency folder are generated under “Dependencies” but those folders are empty.
Some dependencies have a installation.log file - if I see a message here it is mostly something like “make: command not found” - which makes sense as the folders are empty.

This only is valid for dependencies that are installed via “package” not for those via Conda - here everything works fine. Does anyone have a clue whats wrong with Galaxy here?

Many thanks for any input or hint!

Welcome, @schmitts!

Details for fml_gff3togtf

Summary:

  1. Has not been updated since April 2015 by the community contributor author
  2. Does not appear to be maintained anymore
  3. Has several open issues about result content that seem concerning (at least to me)
  4. Uses older dependency handling methods, when used at all
  5. Requires that Biopython and most of the other dependencies are already installed first, before installing the tool
    • Review the repository at the ToolShed and you’ll see this part (“prior install needed”) – this is why those dependencies were not installed as you are probably used to with other tools that have better/more current dependency resolution/installation configuration functions built-in

In short, the ToolShed is open for community contributions. Most are great! Yet some have “gotchas”. Check the last revision date, it is usually a good indicator. Tools can become “not so great” over time if not maintained. Some take more work to install. Some become buggy/incompatible with current Galaxy releases for some reason. And some are abandoned by the original author…

I’d suggest using alternative tools for these manipulations based on the reported open issues alone. But if you really want to use it, you’ll need to install and configure those other dependencies first. For this tool suite, none of this will be automatic. I would also strongly suggest testing it out before allowing end-users to use it, yourself or find an end-user to do that (test for scientifically valid results, based on the specific data expected to be used ongoing). I didn’t review the built-in test cases, but given the open issues, I doubt they cover the content issues reported.

How dependency resolvers are configured by default, plus how to tune. Galaxy releases are also at this RTD site, see “Releases”. https://docs.galaxyproject.org/en/master/admin/dependency_resolvers.html

Adjusting your dep resolvers won’t help with this particular tool since the methods were not incorporated. But the info can help to explain why some tools do not install as expected. It is up to you whether or not it is worth manually working around incomplete dependency configurations any particular tool may have. The original author could also be contacted or an issue reported to their development repository. Sorry, we can’t do more … but tools are owned by the author, and anyone can publish a tool (on purpose).

Example of an alternative tool for a “GFF3-to-GTF” type of manipulation:

  • Tool: gffread Filters and/or converts GFF3/GTF2 records (Galaxy Version 2.2.1.2)
  • ToolShed repo: https://toolshed.g2.bx.psu.edu/repository?repository_id=a0f6caeb94095122
  • The function does have some known limitations. It is just how the 3rd party tool was designed. So give it a test drive first at a public server to see if it does what you need. Then test it at your server as a regression test.

More here

Converting between GFF/GFT/GFF3 annotation format is is tricky... Why?

Both tools and data providers create and release all sorts of hybrid formats. If the GTF version of reference annotation is available from a data provider, definitely use that instead of the GFF3 version to start with, as it will be accepted as an input with more tools. Use “autodetect” for the datatype during Upload to Galaxy and some basic checks will be done. If the expected datatype is not assigned by Galaxy, there is a format issue, and many can be adjusted then the datatype “redetected” until correct (pencil icon > Edit attributes forms > Datatypes tab > Redetect button). Avoid directly assigning a datatype for these data – or expect problems. If Galaxy cannot correctly guess the expected datatype for these three in particular, some formatting problem is still present. True GFF is not the same as GFF3.

These FAQs have more help around the topic:

And here

How to find, request, or create a specific tool

Searching:

There isn’t a one-size-fits-all way to search for tools that perform a specific function, or for existing 3rd party tool wrappers. That said, keyword searches with the original tool name, input/output datatype, or manipulation utility name often work. Try tool panels at UseGalaxy.* servers, or the ToolShed.

Start by reviewing tools available at any of the UseGalaxy.* public servers – all tools are either native to Galaxy itself (installed by default) or sourced/installed from the ToolShed. Check under the tool group “Convert Formats” or use the tool search. Bring up any tool that looks interesting, review the tool form for what it does, then if you want to install it, find the ToolShed link in the “Options” menu at the top of the tool form. If you don’t see a ToolShed link, then the tool is native to Galaxy, and you may need to update your Galaxy version to get it.

Note: Other public Galaxy servers may have tools that are not available from the ToolShed - it depends on how they set it up! - each is independently administered and some contain custom tools. You can always contact them to find out if or where/how the tool you are interested in is available for use/install for a local Galaxy.

UseGalaxy.* and other known Public Galaxy server’s directory listings can be found here: https://galaxyproject.org/use/

Tool development, BioBlend, and the IUC:

If there is a particular function or specific tool that you want to use in Galaxy, and you cannot determine if that has been wrapped already, you can ask here first (or check prior Q&A with a search). We can follow up with the IUC, or you can ask them directly if needed. It could be that the tool/function already exists or is a work-in-progress. The IUC accepts tool wrapping requests. You could also wrap tools yourself, or collaborate with them or others in the community.

Thanks & hope that helps!