bash Cookbook

Forgetting That Pipelines Make Subshells

Problem

You have a script that works just fine, reading input in a while loop:

# This works as expected
COUNT=0
while read PREFIX GUTS
do
    # ...
    if [[ $PREFIX == "abc" ]]
    then
        let COUNT++
    fi
    # ...
done
echo $COUNT

and then you change it to read from a file:

#Don't use; this does NOT work as expected!
COUNT=0
cat $1 | while read PREFIX GUTS
do
    # ... 
    if [[ $PREFIX == "abc" ]]
        then
            let COUNT++
        fi
        # ...
    done
    echo $COUNT   # $COUNT is always '0' which is broken

only now it no longer works…$COUNT keeps coming out as zero.

Solution

Pipelines create subshells. Changes in the while loop do not effect the variables in the outer part of the script, as the while loop is run in a subshell.

One solution: don’t do that (if you can help it). In this example, instead of using cat to pipe the file’s content into the while statement, you could use I/O redirection to have the input come from a redirected input rather than setting up a pipeline:

# Avoid the | and sub-shell; use "done < $1" instead
# It now works as expected
COUNT=0
while read PREFIX GUTS
do
    # ...
  if [[ $PREFIX == "abc" ]]
   then

            let COUNT++

        fi

        # ...

done < $1 # <<<< This is the key line

echo "$COUNT now lives in the main script"

Such a rearrangement might not be appropriate for your problem, in which case you’ll have to find other techniques.

Discussion

If you add an echo statement inside the while loop, you can see $COUNT increasing, but once you exit the loop, $COUNT will be back to zero. The way that bash ...