Hack University - Capstone: automate a real workflow

The scenario

You're on-call for a small app. Every day, log files land in project/logs/, and your job each morning is to produce project/reports/summary.txt: one line per log file with its error count.

Done by hand, that's opening every file and counting. Done your way, it's a script that uses one tool from nearly every unit:

mkdir -p, paths (unit 2)
writing files with > and >> (units 3, 5)
grep -c (unit 4)
a for loop over a wildcard (units 4, 9)
a function with $1 (units 8, 9)
redirecting a whole loop's output into a file (unit 5)

First, set the stage.

The capstone data flow. The wildcard expands the logs directory into a list of two files, the loop calls the report function once per file so each pass prints one summary line, and the redirection placed after done collects the whole loop into summary.txt. Because the file is opened once by the loop rather than appended to inside it, each run produces a fresh and complete report.

Setting the stage

Two directories and two log files with known contents, so the counts later are verifiable: mon.log contains one error and tue.log contains two. The final ls confirms both files are in place.

mkdir -p project/logs project/reports
echo "INFO deploy ok" > project/logs/mon.log
echo "ERROR timeout" >> project/logs/mon.log
echo "INFO backup ok" > project/logs/tue.log
echo "ERROR disk full" >> project/logs/tue.log
echo "ERROR retry failed" >> project/logs/tue.log
ls project/logs

Output

mon.log
tue.log

Nothing in this setup is new. It is mkdir -p from unit 2, echo with the > and >> arrows from units 3 and 5, and ls given a path so the shell never has to move.

The full report script

Everything converges here. A report function prints one summary line for the file it is given, a loop runs it over every .log in project/logs, the entire loop's output is redirected into project/reports/summary.txt, and cat displays the finished report.

mkdir -p project/logs project/reports
echo "INFO deploy ok" > project/logs/mon.log
echo "ERROR timeout" >> project/logs/mon.log
echo "INFO backup ok" > project/logs/tue.log
echo "ERROR disk full" >> project/logs/tue.log
echo "ERROR retry failed" >> project/logs/tue.log
report() {
  echo "$1: $(grep -c ERROR "$1") errors"
}
for f in project/logs/*.log; do
  report "$f"
done > project/reports/summary.txt
cat project/reports/summary.txt

Output

project/logs/mon.log: 1 errors
project/logs/tue.log: 2 errors

The three ideas doing the work

The function body is a single echo built around command substitution from lesson 9-3: echo "$1: $(grep -c ERROR "$1") errors" drops the count into the middle of a sentence.
An entire loop can be redirected. Putting > project/reports/summary.txt after done captures every iteration's output, rather than needing a redirect inside the body.
The counts match the setup, one error for Monday and two for Tuesday, which is how you know the pipeline is correct rather than merely quiet.

Where you go from here

You now hold the everyday toolkit of a working engineer: navigate anywhere, inspect anything, chain small tools into answers, and script away repetition. Three natural next steps:

Git & GitHub: version control lives entirely in this terminal you now speak fluently.
Use the shell daily. Take the long way on purpose for two weeks: mkdir instead of right-click, grep instead of scrolling.
When a task feels repetitive, write the loop. Every script you write pays rent forever.

Putting the redirection once after done, as done > project/reports/summary.txt, is better than appending with >> inside the function for two related reasons.

A single redirection opens the file once and captures the whole loop, so the report is written as one coherent unit. It also starts from a clean file every run, because > truncates before writing. Appending inside the loop with >> would instead pile today's lines on top of yesterday's, and the file would grow stale and misleading without ever looking broken.

Choosing where to redirect, rather than just which arrow to use, is real script-craft. It is the difference between a report that is always current and one that quietly accumulates garbage.