Batch input workflow - how to send jobs to distinct histories?

Hi @kam132132

Yes, you found the available option. Everything can be written to a new history and tagged. Then you can copy datasets into new histories if you want to later. What you see in a history as a “dataset” is just a pointer to the actual dataset – so you can have as many copies as you want across histories, and it is all still just that original file (and only counted once against any quota limits, or storage location).

One workflow invocation + one workflow version + one output history

  • All the original inputs are mapped over into that output history if it is different than the starting data history. This is intentional. Together, these function as a reproducibility “log” for what happened with all the details.
  • Tagging the original samples will propagate that tag down into all the outputs.
  • Search the history with a tag (or click on that tag on any dataset) to filter that history view for just the associated data with that same tag.

If the reason you want to split up the outputs was for data storage reasons, you can split where the data is stored between storage locations when executing the workflow. We have a topic here with some screenshots can explain how to use that function. → Result files were deposited in different storage (from 30-day storage to permanent storage) - #2 by jennaj

One more option: If you are running large batches, you could execute the workflow as a batch. Run it once per sample in a loop. You would need to use the API for this custom implementation. All the results would write back into a history for application review or sharing for later. You can run jobs using the API against public servers with your standard account.

Hope this helps! :slight_smile: