Redirection: sending output to files

Since lesson 3-1 you have been writing files with echo and two different arrows. The difference is that > overwrites the file while >> appends to the end of it. One arrow replaces the whole contents, two arrows add to what is already there, and both create the file if it does not exist yet.

This lesson explains why that works. Every command has redirectable output streams, and the arrows are just the syntax for re-plugging them.

Every command has three streams

Redirection is what turns commands into saved artifacts: a report written to a file instead of scrolling past, an overnight job's errors captured for morning reading, a program fed test input from a file instead of retyped by hand. To re-route output like that, you first need to know what the routes are.

When any program runs, the OS hands it three data channels:

Redirection re-plugs these channels:

• command > file: stdout goes into the file (overwrite)
• command >> file: stdout appends to the file
• command < file: stdin comes from the file instead of the keyboard
• command 2> file: stderr (channel 2!) goes into the file

Normal results and error messages travel on separate channels. That's why you can save one and still see the other.

The overwrite arrow, in action

Two lines go into diary.txt, and then a single > wipes both out and replaces them with one new line.

echo "first" > diary.txt
echo "second" >> diary.txt
echo "REPLACED" > diary.txt
cat diary.txt

Output

REPLACED

The file holds exactly one line at the end. Because the third echo used > rather than >>, the redirection truncated the file to empty before writing, so first and second are gone with no way to get them back.

Capturing an error with 2>

Errors travel on stream 2, separately from results. Here cat fails because missing.txt does not exist, but 2> catches the complaint in a file so nothing appears on screen, and the session continues. The final cat reads the captured message back.

cat missing.txt 2> errors.txt
echo "still running"
cat errors.txt

Output

still running
cat: missing.txt: No such file or directory

The order of that output is the interesting part. still running prints first, even though the failure happened before it, because the error message went into a file instead of to the screen and only surfaced later when errors.txt was read. Without the 2>, the complaint would have appeared immediately, ahead of still running.

Throwing output away: /dev/null

/dev/null is a special file provided by the OS whose only job is to discard everything written to it. Redirecting a stream there is how you say the output genuinely does not matter. For instance, ls *.txt 2> /dev/null shows whatever exists and silently drops the complaint when nothing matches, and scripts use this constantly to keep their output clean.

One more combination shows up everywhere in the wild: command > out.log 2>&1. The first part sends stdout to the file, and 2>&1 then sends stderr to wherever stream 1 is currently going, so both streams land in the same file. That is how an unattended job's whole story, results and errors together, ends up in a single log.

Discarding just the error, keeping the result

ls is asked about two files, only one of which exists. The complaint about missing.txt is discarded into /dev/null, the genuine result still prints, and the session carries on normally.

echo "hello" > exists.txt
ls exists.txt missing.txt 2> /dev/null
echo "script still running"

Output

exists.txt
script still running

This is the payoff of having two separate output channels. Stream 1 carried exists.txt to the screen while stream 2 carried the error into the void, and neither redirection interfered with the other. Removing the 2> /dev/null would bring the error message back without changing the rest of the output.

Adding 2> /dev/null to a command discards the command's error output. The 2> selects stream 2, which is stderr, and /dev/null throws away whatever is written to it.

The consequence is that errors vanish while normal output on stream 1 still reaches the screen. Discarding the normal output instead is the mirror-image spelling, > /dev/null, which leaves error messages visible.

Both arrows appear together in this short sequence. seq 1 5 > five.txt creates the file with five numbers, the >> on the next line adds a word to the end without disturbing them, and tail reads the last two lines to show the seam.

seq 1 5 > five.txt
echo "done" >> five.txt
tail -n 2 five.txt

Output

5
done

The append arrow is what makes this work. Using > on the second line would truncate the file first, erasing all five numbers and leaving done as the only line, and tail would then have just one line to print.