Add a header line to concatenated files

Hi, I’m writing a tool to concatenate output csv files from a custom Rmarkdown galaxy analysis tool and am having some issues. I borrowed the concatenation code from the “concatenate_multiple_datasets” tool, which works just fine to concatenate files. The problem is, I really just want to append the concatenated dataset to a header line, but whenever I try to do this it looks like Galaxy decides that we’re just going to concatenate file names to the header.

Here’s the code, where “some stuff” will eventually be a megalist of comma separated column IDs:

  echo "some stuff" > $out_file1
  #for $file in $input
  #end for
  >> $out_file1


And here’s the output:

some stuff cat /mnt/d/git_repos/Galaxy/galaxy/database/objects/c/f/d/dataset_cfda7470-0c6e-4ff1-8e63-1fd8a9bdffd0.dat /mnt/d/git_repos/Galaxy/galaxy/database/objects/a/3/9/dataset_a397895a-57b5-4963-828f-58414280225c.dat

Here’s what Galaxy says it’s doing on the command line, which looks right to me?:

echo “some stuff” > /mnt/d/git_repos/Galaxy/galaxy/database/objects/4/e/6/dataset_4e657268-daa7-4a2a-9c37-c04d7fd4f221.dat
cat /mnt/d/git_repos/Galaxy/galaxy/database/objects/c/f/d/dataset_cfda7470-0c6e-4ff1-8e63-1fd8a9bdffd0.dat /mnt/d/git_repos/Galaxy/galaxy/database/objects/a/3/9/dataset_a397895a-57b5-4963-828f-58414280225c.dat >> /mnt/d/git_repos/Galaxy/galaxy/database/objects/4/e/6/dataset_4e657268-daa7-4a2a-9c37-c04d7fd4f221.dat

If I leave out the echo "some stuff" > $out_file1 line everything just concatenates as normal. I’m a beginner at Cheetah and python so may have just missed something obvious (at least, I hope this is obvious). Any advice would be appreciated!

I strongly recommend you do not write a custom tool for this. You are creating a burden on yourself and anyone else that might use this tool to maintain it.

You can readily accomplish this task by simply pre-creating a dataset with just the header and adding it to the top of your collection of CSVs. The existing cat tool will then concatenate the header along with your csv data.

Hi @KeelyDulmage,
your immediate issue is a missing & between your shell commands.
Beyond that I agree with @innovate-invent. If you write this just to understand how Galaxy tool development works that’s perfectly fine, but otherwise don’t reinvent the wheel :slight_smile:

1 Like

Thanks wm75, that was the problem.
I understand your and innovate-invent’s POVs, and will take them into account. In this case, this will be purely downstream of a custom tool which is absolutely not going in the toolshed, so we are stuck with some default amount of tool maintenance anyway. But I will see if the simple option is acceptable to the end users.

I would not use a single & between the commands, that will create a small race condition between the first and second commands. Use && between the commands.

1 Like