GAT step-12 - Testing tool error caching job destination dynamic rule


I am trying to follow the GTN Archive 2023-11-01 tutorial and resumed from Step-12 about Mapping Jobs to Destinations using TPV.
All the beginning up to Job Resource Selectors seems OK (i.e., the Testing Tool works for admin and fails for user).
After that there are many changes made (with no check point in between) and eventually running again the Testing Tool with various resource parameters systematically fails, indicating:

An error occurred with this dataset:
format txt database ?
Encountered an unhandled exception while caching job destination dynamic rule.

I cannot see any obvious difference between the reference files and mine (other than that tpv_rules_local.yml includes for instance a section about pulsar which is not mentioned in the tutorial).

Can someone provide working reference files for that step and/or help me sort mine out?

Thanks in advance

Hi @j34ni

It looks like the tutorial was updated since the archive. In the tutorial map, Pulsar is covered in the next tutorial. Mapping Jobs to Destinations using TPV

Please give that a review, and we can follow up is something is still missing.

Hi @jennaj

I still get the same error at the same stage, until then everything worked as expected, but now whatever the resource parameters the job fails, no slurm job is submitted and hence there is no way to do scontrol show job xx, is there?

Have you tried it recently and does everything behave as described in the tutorial?

Hello @j34ni

I think those processes changed recently, so I asked at the Dev chat for help. You could also join and followup if you want to. You're invited to talk on Matrix

The version of Galaxy probably matters. Not sure if the OS matters. Maybe at least post that into the chat for context?

I tried various combinations of OS and Python, and the one which took me farthest in this GAT is: Ubuntu 20.04.6 LTS (Focal Fossa) and Python 3.8.10

As for the version of Galaxy it is release_23.0 (as set in step-1) and ansible is core 2.13.13

@j34ni it would help if you could share your config. Or revert your changes until it works again and share the snippet that broke it.