Dear Galaxy community,
I’m new to tool wrapping, and trying to wrap a Python code (RFDiffusion) that uses Hydra (https://hydra.cc/) for configuration, rather than setting parameters with command line flags.
Just posting a quick call-out to see if anyone has worked with Hydra config in a galaxy tool before?
I see it being adopted by more Machine Learning codes lately, because it allows composition from hierarchical config files (build specific cases on top of base.yaml) with occasional CLI overrides from the user.
It makes composition pretty simple:
|
|- configuration
| |
| |- base.yaml
| |-additional_conf.yaml
|
|_run-inference.py
Where base.yaml does your config and gets overwritten
Base inference Configuration.
inference:
input_pdb: null
num_designs: 10
…
I’ve been helpfully pointed to tool > configfiles > configfile in planemo as a way to set parameters from a temporary file.
However I wanted to check if anyone has prior experience before embarking on trying to shoehorn a composable markup config system that way, since it feels like I need more permance of my Hydra files than my assumption of the configfile wrapped in XML being temporary and not overridable?
Many thanks!
Keiran
Hi @keiran-rowell-unsw!
Thanks for reaching out! The “configfile” feature in Galaxy is a very dynamic way to create config-files on the fly and pass it to the CLI.
A few tools create python scripts on the fly with the “configfile” feature. Its essentially a template that you can fill with user-provided input.
See many examples here: Code search results · GitHub
I think its also the way to go for Hydra. In the end you need to collect user-inputs via a form. If you put those inputs into a JSON, Hydra-YAML or create a script in the fly doesn’t matter much I think.
You could for example also offer a conditional to the user, ask for pre-defined models, offer then specific options to fill and depending on the conditional + the specific values you fill one of many Hydra templates.
Happy to help you if you like, find me on Matrix,
Bjoern
2 Likes
Hi @bjoern.gruening thanks for responding!
It’s quite possible that I can fill most of the sensible defaults in the configfile feature. I’ve looked at a few examples on the IUC tools and they don’t seem to inherit other configs like I’d ideally like, but that might be me not finding similar examples.
The good news is I have a colleague from Aus BioCommons willing to showing me the ropes in wrapping and thinks he can get some of the Hydra features mimiced by using JSON with switch cases.
The user-presented conditional for pre-trained Deep Learning models would definitely whittle down the cases, nuc. acids will displace a different set of veritable variables than proteins. That’s a great suggestion.
Thanks. What we’ll do next is try our own attempts over the next few weeks, and then reach out on Matrix if/when we discover what works and what are the pain points.
Best,
Keiran
1 Like
Sounds great, let me know if you need any help!