Fields and Records

Records and fields

awk splits input into records on RS (a newline by default), then splits each record into fields on FS (runs of blanks by default).

printf 'alice 30 nyc\nbob 41 sf\n' | awk '{ print $3, $1 }'
# nyc alice
# sf bob

Field numbers are expressions, so $NF, $(i+1), and $x all work. Referring to a field past NF yields the empty string and does not change NF.

Setting the field separator

The default FS of " " is special: it splits on runs of spaces, tabs, and newlines and strips leading and trailing blanks. Any other value, including a one-character regex, turns off that behavior.

printf 'a  b\n' | awk -F' '   '{print NF}'   # 2, runs collapse
printf 'a  b\n' | awk -F'[ ]' '{print NF}'   # 3, empty middle field

Setting FS inside a normal rule takes effect on the next record, not the current one. Set it in BEGIN or with -F.

Assigning to a field rebuilds the record

Assigning to $n rewrites $0, joining the fields with OFS. Assigning to $0 re-splits into fields with FS.

# strip column 2, then print with the new separator
printf 'a b c\n' | awk 'BEGIN{OFS="-"} { $2="X"; print }'
# a-X-c

# the classic force-a-rebuild idiom: touch $1 with a no-op
printf 'a b c\n' | awk 'BEGIN{OFS=","} { $1=$1; print }'
# a,b,c

Without that $1=$1, print emits the untouched original $0 and your OFS looks ignored. This is the single most common awk surprise.

Growing and shrinking a record

printf 'a b\n' | awk '{ $(NF+1)="c"; print }'    # a b c
printf 'a b c d\n' | awk '{ NF=2; print }'       # a b
printf 'a b c\n' | awk '{ $5="e"; print; print NF }'
# a b c  e
# 5

• Assigning a larger NF pads with empty fields, a smaller one truncates.
• Assigning past NF fills the gap with empty fields and bumps NF.
• Both rebuild $0 with OFS.

Multi-line and custom records

# paragraph mode: records are blank-line separated, fields split on
# newlines as well as FS
awk 'BEGIN{RS=""} { print "record", NR, "has", NF, "fields" }' notes.txt

# one record per null byte, for find -print0 style input (gawk and mawk,
# macOS awk collapses RS="\0" into paragraph mode)
find . -name '*.log' -print0 | gawk 'BEGIN{RS="\0"} { print NR, $0 }'

# split on a regex (gawk, mawk, and current macOS awk, not POSIX)
printf 'a1b22c\n' | awk 'BEGIN{RS="[0-9]+"} { print NR": "$0 }'

POSIX only guarantees the first character of RS is used, so a regex RS is an extension. gawk, mawk, and current macOS awk all support it, an older or minimal awk may take only the first character. In gawk, RT holds the text that matched RS.

Reading many files

FILENAME is the current file, FNR resets per file, NR does not. That pairing drives the two-file join idiom.

# print each file's line count without wc
awk 'END { print FILENAME, FNR }' a.txt

# process file1 fully, then file2
awk 'NR==FNR { seen[$1]; next } $1 in seen' keys.txt data.txt

FILENAME is empty inside BEGIN, and it stays "" (not -) while reading stdin on both macOS awk and gawk. In END it still holds the last file read.