awk Cheatsheet

Program Structure

Use this awk reference while you build software engineering projects, review code, or refresh the syntax you reach for most.

The pattern action model

An awk program is a list of pattern { action } rules. For every input record awk tests each pattern in order and runs the action of every pattern that matches.

awk 'pattern { action } pattern { action }' file...

Pattern with no action means { print } (print the whole record).
Action with no pattern runs on every record.
Rules are separated by newlines or ;, statements inside an action by newlines or ;.

# these three are equivalent
awk '/error/' app.log
awk '/error/ { print }' app.log
awk '/error/ { print $0 }' app.log

BEGIN and END

BEGIN runs once before the first record is read, END once after the last. Neither is required, and you may have several of each (they run in source order).

awk 'BEGIN { FS=","; total=0 }
     NR > 1  { total += $3 }
     END     { printf "rows=%d total=%.2f\n", NR-1, total }' sales.csv

- A program made only of BEGIN blocks never reads input at all, so awk 'BEGIN{print 2^10}' works with no file and no stdin. - If any rule outside BEGIN exists, awk reads stdin when no file is given. - END still sees the last record in $0, NR, and NF.

Invocation forms

Form	Meaning
`awk 'prog' file...`	program as the first argument
`awk -f prog.awk file...`	program from a file, repeatable
`awk -F: 'prog' file`	set `FS` to `:`
`awk -v n=5 'prog' file`	set variable `n` before `BEGIN`
`awk 'prog' n=5 file`	set `n` when awk reaches that argument
`awk 'prog' -`	read stdin explicitly

# a program file, with a shebang
cat > sum.awk <<'EOF'
#!/usr/bin/awk -f
{ s += $1 }
END { print s }
EOF
chmod +x sum.awk && ./sum.awk numbers.txt

-v assignments happen before BEGIN. Bare var=value operands are applied when awk walks past them in the argument list, so they can carry a per-file setting.

macOS gotcha. On the bundled awk (20200816) a second FS= operand updates the variable but the records keep splitting on the previous separator. gawk gets this right. If you need two separators, run two awks and pipe them.

Comments and quoting

# a comment runs to end of line
{ print $1 }   # trailing comments are fine too

Wrap the program in single quotes so the shell leaves $1 and \n alone. To use a literal single quote inside, either close and reopen the quoting or pass the character in with -v.

awk -v q="'" '{ print q $0 q }' file
awk '{ print "it'"'"'s" }' file      # ugly but portable

Exit status and errors

exit in an action jumps straight to END, then leaves. exit N sets the status.
exit inside END leaves immediately.
Runtime errors go to stderr and exit non-zero.

awk '/FATAL/ { found=1; exit } END { exit found ? 0 : 1 }' app.log

Which awk am I running

Three families are common and they are not interchangeable.

Implementation	Typical home	Check
`gawk` (GNU awk)	Linux, `brew install gawk`	`awk --version` prints `GNU Awk 5.x`
`bwk` / one true awk	macOS, BSD	`awk --version` prints `awk version 20200816`
`mawk`	Debian and Ubuntu default	`awk -W version`

Everything in this reference is POSIX awk unless a row says gawk only. On macOS the shipped awk has no systime, strftime, gensub, asort, PROCINFO, IGNORECASE, or switch, so scripts that use them need brew install gawk and the gawk name.