jq Cheatsheet

Reduce and Group

Use this jq reference while you build software engineering projects, review code, or refresh the syntax you reach for most.

`reduce`

reduce folds a stream into a single value. It is the only way to accumulate state in jq, and the syntax has no shortcut.

reduce GENERATOR as $var (INIT; UPDATE)

Inside UPDATE, . is the accumulator so far and $var is the current element. The result is one output, the final accumulator.

jq -c 'reduce .items[] as $i (0; . + $i.price)'  data.json   # 60
jq -cn 'reduce (1,2,3) as $x (0; . + $x)'                    # 6
jq -cn 'reduce empty as $x (0; . + $x)'                      # 0, INIT survives
jq -cn 'reduce range(1;6) as $n (1; . * $n)'                 # 120, factorial
jq -c 'reduce .items[] as $i ({}; .[$i.cat] += [$i.id])' data.json
# {"x":[1,3],"y":[2]}
jq -c 'reduce .items[].cat as $c ({}; .[$c] += 1)'  data.json
# {"x":2,"y":1}

- Forgetting the (INIT; UPDATE) parens is the most common syntax error. There is no one-argument form. - += on a missing key works because null + [x] is [x] and null + 1 is 1, so histograms and grouped lists need no initialization. - The generator can be any multi-output filter, including range, inputs, .., and a destructuring pattern.

jq -c 'reduce .items[] as {$id, $price} ({}; .[$id|tostring] = $price)' data.json
# {"1":10,"2":20,"3":30}

`foreach`

foreach is reduce that emits a value at every step instead of only at the end. That makes it the tool for running totals, running maxima, and numbering.

foreach GENERATOR as $var (INIT; UPDATE)
foreach GENERATOR as $var (INIT; UPDATE; EXTRACT)

# two-arg: emit the accumulator after each step
jq -c '[foreach .items[] as $i (0; . + $i.price)]'  data.json   # [10,30,60]
jq -c '[foreach .items[] as $i (0; . + 1)]'         data.json   # [1,2,3]

# three-arg: emit EXTRACT, so the state can be richer than the output
jq -c '[foreach .items[] as $i (0; . + $i.price;
        {id: $i.id, running: .})]' data.json
# [{"id":1,"running":10},{"id":2,"running":30},{"id":3,"running":60}]

# running maximum
jq -c '[foreach .items[].price as $p (0;
        if $p > . then $p else . end)]' data.json   # [10,20,30]

	Outputs	Use for
`reduce`	exactly one, the final state	totals, merges, lookup tables
`foreach`	one per input element	running values, numbering, deltas

`group_by`

group_by(f) returns an array of arrays, grouping consecutive elements with an equal f. It sorts by f first, so no prior sort_by is needed and the groups come back in sorted key order.

jq -c '.items | group_by(.cat)' data.json
# [[{"id":1,...,"cat":"x"},{"id":3,...,"cat":"x"}],[{"id":2,...,"cat":"y"}]]

jq -cn '[{c:"z"},{c:"a"},{c:"z"}] | group_by(.c) | map(.[0].c)'   # ["a","z"]

The idiom is always group_by then map over the groups, because the raw nested array is rarely what you want.

# count per group
jq -c '[.items[].cat] | group_by(.) | map({key: .[0], value: length})
       | from_entries' data.json                     # {"x":2,"y":1}

# aggregate per group
jq -c '.items | group_by(.cat)
       | map({cat: .[0].cat, n: length, total: (map(.price) | add)})' data.json
# [{"cat":"x","n":2,"total":40},{"cat":"y","n":1,"total":20}]

# group by a computed key, groups come back in sorted key order
jq -c '.items
       | group_by(if .price > 15 then "high" else "low" end)
       | map({bucket: (if .[0].price > 15 then "high" else "low" end),
              ids: map(.id)})' data.json
# [{"bucket":"high","ids":[2,3]},{"bucket":"low","ids":[1]}]

A plain reduce is often shorter and always faster than group_by for counting, because it makes one pass and never sorts. Use group_by when you need the whole group, not just a tally.

Sorting

Filter	Returns
`sort`	the array sorted by jq's total type order
`sort_by(f)`	sorted by `f`, stable
`sort_by(f, g)`	sorted by `f` then `g` as a tiebreak, a comma not a semicolon
`unique`	sorted and deduplicated
`unique_by(f)`	one element per distinct `f`, the first after sorting
`min_by(f)` / `max_by(f)`	the element itself, not the key
`reverse`	reversed, no sorting

jq -c '.items | sort_by(.price) | map(.id)'      data.json   # [1,2,3]
jq -c '.items | sort_by(-.price) | map(.id)'                 # [3,2,1]
jq -c '.items | sort_by(.name) | reverse | map(.name)'       # ["c","b","a"]
jq -c '.items | sort_by(.cat, -.price) | map([.cat,.price])'
# [["x",30],["x",10],["y",20]]
jq -c '.items | unique_by(.cat) | map(.id)'                  # [1,2]
jq -c '.items | min_by(.price).id, max_by(.price).id'        # 1 then 3
jq -cn '[3,1,2,1] | sort, unique, reverse'

- sort_by(-.x) only works for numbers. For a descending string sort use sort_by(.x) | reverse. - There is no sort_by/2. Multiple keys go in one argument separated by a comma, as sort_by(.a, .b). - sort_by, group_by, and unique_by all sort with jq's total order, so null sorts first. A missing key therefore floats to the top. - min_by and max_by return the whole element. min and max on a plain array return the value.

Lookup tables with `INDEX`

INDEX builds an object keyed by an expression, which turns a repeated select(.id == x) scan into a single hash lookup.

jq -c 'INDEX(.items[]; .id | tostring) | keys' data.json     # ["1","2","3"]
jq -c '.items | INDEX(.id | tostring) | ."2".name' data.json # "b"
jq -cn '[{id:"a"},{id:"b"}] | INDEX(.id)'
# {"a":{"id":"a"},"b":{"id":"b"}}

Form	Keys from
`INDEX(stream; keyf)`	`keyf` applied to each value of `stream`
`INDEX(keyf)`	`keyf` applied to each element of the input array
`IN(source)`	`true` when the input appears in `source`
`IN(src; s)`	`true` when any output of `src` appears in `s`

jq -c '[.items[] | select(.cat | IN("x"))] | length'      data.json   # 2
jq -c '[.items[] | select(IN(.cat; "x","y"))] | length'   data.json   # 3

INDEX stringifies a non-string key for you, so INDEX(.id) on numeric ids gives {"1":...,"2":...}. from_entries does not, it raises Cannot use number as object key, so pipe the key through tostring when you build the object by hand.

Joining two documents

There is no join on objects, so the pattern is: index one side, then walk the other. This is the single most useful thing reduce and INDEX do together.

# prices.json: [{"id":1,"cost":4},{"id":2,"cost":9}]
jq -c --slurpfile p prices.json '
  ($p[0] | INDEX(.id | tostring)) as $by_id
  | .items
  | map(. + {cost: ($by_id[.id | tostring].cost // null)})
' data.json

# sum a nested structure without flattening it first
jq -c 'reduce (.. | numbers) as $n (0; . + $n)' data.json     # 69

# rebuild a document from a path stream
jq -c 'reduce (tostream | select(length == 2)) as [$p,$v]
       ({}; setpath($p; $v))' data.json

add(f) is the one-argument shorthand for the summing case, added in jq 1.7:

jq -c 'add(.items[].price)' data.json    # 60
jq -cn '[{a:1},{a:2}] | add(.[].a)'      # 3

Reduce and Group

reduce

foreach

group_by