Using HERE documents to parameterize input files

tags: modeling programming (April 28, 2008)

When working with the shell, one often passes input files to interpreters like awk, ode, or others. Often, one might want to parameterize such input files. For example, instead of calling

$ ode < example.ode

one might actually want to pass in parameters to set some values in example.ode on the fly. What one seeks for is something similar to

$ cat example.ode | param 25 > example.dat

If only param would exist... This is where HERE documents become handy.

HERE documents

This less prominent form of input redirection allows you to place the contents you would otherwise put into a separate file right at the place where you would read in that file. It is best explained with an example. Assume you have a file named input.txt containing the line

content of input file

and some script that uses the input redirection

sed < input.txt

then HERE documents allow you to simply embed the input file into the script, writing

sed <<DELIMITER
content of the input file
DELIMITER

where DELIMITER can be an arbitrary string.

cat scripts

cat scripts are shell scripts that simply cat a HERE document to stdout. The advantage of this construct above 'passive' files is that you can do parameter substitution. And this is the mechanism that allows you to parameterize said input files. Consider the file input.txt that actually is a cat script:


#! /bin/sh
cat <<DELIMITER
[put your template $1 here]
DELIMITER

To parameterize it, use the following call:

$ sh input.txt text

which will result in

[put your template text here]

The shell provides up to 9 parameters that you can pass in as arguments and refer to as $1 to $9. $0 will be replaced by the name of the input file. Of course, you can use more sophisticated constructs using more sophisiticated variable substitutions.