Tool wrapping with Hydra config files -- not CLI arguments -- driving the Python code

1

Dear Galaxy community,

I’m new to tool wrapping, and trying to wrap a Python code (RFDiffusion) that uses Hydra (https://hydra.cc/) for configuration, rather than setting parameters with command line flags.

Just posting a quick call-out to see if anyone has worked with Hydra config in a galaxy tool before?

I see it being adopted by more Machine Learning codes lately, because it allows composition from hierarchical config files (build specific cases on top of base.yaml) with occasional CLI overrides from the user.

It makes composition pretty simple:
|
|- configuration
| |
| |- base.yaml
| |-additional_conf.yaml
|
|_run-inference.py

Where base.yaml does your config and gets overwritten

Base inference Configuration.

inference:
input_pdb: null
num_designs: 10
…

I’ve been helpfully pointed to tool > configfiles > configfile in planemo as a way to set parameters from a temporary file.

However I wanted to check if anyone has prior experience before embarking on trying to shoehorn a composable markup config system that way, since it feels like I need more permance of my Hydra files than my assumption of the configfile wrapped in XML being temporary and not overridable?

Many thanks!
Keiran

2

Hi @keiran-rowell-unsw!

Thanks for reaching out! The “configfile” feature in Galaxy is a very dynamic way to create config-files on the fly and pass it to the CLI.

A few tools create python scripts on the fly with the “configfile” feature. Its essentially a template that you can fill with user-provided input.

See many examples here: Code search results · GitHub

I think its also the way to go for Hydra. In the end you need to collect user-inputs via a form. If you put those inputs into a JSON, Hydra-YAML or create a script in the fly doesn’t matter much I think.

You could for example also offer a conditional to the user, ask for pre-defined models, offer then specific options to fill and depending on the conditional + the specific values you fill one of many Hydra templates.

Happy to help you if you like, find me on Matrix,
Bjoern