Add a header line to concatenated files

1

Hi, I’m writing a tool to concatenate output csv files from a custom Rmarkdown galaxy analysis tool and am having some issues. I borrowed the concatenation code from the “concatenate_multiple_datasets” tool, which works just fine to concatenate files. The problem is, I really just want to append the concatenated dataset to a header line, but whenever I try to do this it looks like Galaxy decides that we’re just going to concatenate file names to the header.

Here’s the code, where “some stuff” will eventually be a megalist of comma separated column IDs:

    <command><![CDATA[
  echo "some stuff" > $out_file1
  cat
  #for $file in $input
      ${file}
  #end for
  >> $out_file1

    ]]>
</command>

And here’s the output:

some stuff cat /mnt/d/git_repos/Galaxy/galaxy/database/objects/c/f/d/dataset_cfda7470-0c6e-4ff1-8e63-1fd8a9bdffd0.dat /mnt/d/git_repos/Galaxy/galaxy/database/objects/a/3/9/dataset_a397895a-57b5-4963-828f-58414280225c.dat

Here’s what Galaxy says it’s doing on the command line, which looks right to me?:

echo “some stuff” > /mnt/d/git_repos/Galaxy/galaxy/database/objects/4/e/6/dataset_4e657268-daa7-4a2a-9c37-c04d7fd4f221.dat
cat /mnt/d/git_repos/Galaxy/galaxy/database/objects/c/f/d/dataset_cfda7470-0c6e-4ff1-8e63-1fd8a9bdffd0.dat /mnt/d/git_repos/Galaxy/galaxy/database/objects/a/3/9/dataset_a397895a-57b5-4963-828f-58414280225c.dat >> /mnt/d/git_repos/Galaxy/galaxy/database/objects/4/e/6/dataset_4e657268-daa7-4a2a-9c37-c04d7fd4f221.dat

If I leave out the echo "some stuff" > $out_file1 line everything just concatenates as normal. I’m a beginner at Cheetah and python so may have just missed something obvious (at least, I hope this is obvious). Any advice would be appreciated!

2

I strongly recommend you do not write a custom tool for this. You are creating a burden on yourself and anyone else that might use this tool to maintain it.

You can readily accomplish this task by simply pre-creating a dataset with just the header and adding it to the top of your collection of CSVs. The existing cat tool will then concatenate the header along with your csv data.

3

Hi @KeelyDulmage,
your immediate issue is a missing & between your shell commands.
Beyond that I agree with @innovate-invent. If you write this just to understand how Galaxy tool development works that’s perfectly fine, but otherwise don’t reinvent the wheel :slight_smile:

1 Like
4

Thanks wm75, that was the problem.
I understand your and innovate-invent’s POVs, and will take them into account. In this case, this will be purely downstream of a custom tool which is absolutely not going in the toolshed, so we are stuck with some default amount of tool maintenance anyway. But I will see if the simple option is acceptable to the end users.

5

I would not use a single & between the commands, that will create a small race condition between the first and second commands. Use && between the commands.

1 Like