A Tiny Test Runner

One failure should not hide the rest

Plain asserts have a flaw: the program stops at the first failure, so you never learn whether the other tests pass. Real test tools like pytest and unittest solve this by running each test separately and printing a report.

We can build the heart of such a test runner in ten lines. The idea:

1. write each test as a small function whose name starts with test_
2. loop over the test functions
3. call each one inside try/except AssertionError (exception handling from Advanced Python)
4. print PASS or FAIL with the test's name, then a summary

Building it once demystifies every test framework you will ever meet. They all do this, plus conveniences.

A ten-line test runner

The runner loops over test functions, calls each one inside a try, and prints a line per test plus a summary. test.__name__ is a function attribute holding the function's own name, which gives readable output for free.

def add(a, b):
    return a + b

def test_adds_positives():
    assert add(2, 3) == 5

def test_adds_negatives():
    assert add(-2, -3) == -5

def test_zero():
    assert add(0, 0) == 0

def run_tests(tests):
    passed = 0
    for test in tests:
        try:
            test()
            print("PASS", test.__name__)
            passed += 1
        except AssertionError:
            print("FAIL", test.__name__)
    print(f"{passed}/{len(tests)} passed")

run_tests([test_adds_positives, test_adds_negatives, test_zero])

Output

PASS test_adds_positives
PASS test_adds_negatives
PASS test_zero
3/3 passed

The list passed to run_tests holds the functions themselves, not calls to them. Writing [test_zero] puts the function object in the list, while [test_zero()] would call it immediately and put its return value there, which is a distinction worth being deliberate about.

passed += 1 sits after the call rather than before it, and that ordering is what makes the count honest. A failing test raises before reaching that line, so it never gets counted.

The naming convention of a test_ prefix is doing nothing here yet, since the tests are listed by hand. Real runners use it to find tests automatically by scanning a module, which is the one convenience this version lacks most.

Why each test runs inside try and except

The runner wraps each call in try / except AssertionError so that one failing test is recorded as a FAIL and the run still reaches the remaining tests.

Without the except, the first AssertionError would propagate out of run_tests and crash the whole program, so you would only ever learn about the first failure. Catching it lets the runner record the result and keep going, which means one run tells you the status of every test.

That difference matters more than it sounds. Ten failing tests that share one root cause look very different from ten failures scattered across unrelated features, and you cannot tell those apart if you only ever see the first one.

This is exactly what pytest does at its core. Everything else it offers, including test discovery, detailed failure output, fixtures, and parameterization, is built on top of this same run-each-test-in-isolation loop.

Letting the runner find a real bug

multiply below is implemented with the wrong operator, and the tests catch it. Run it as written to see the failures first.

def multiply(a, b):
    return a + b

def test_multiplies():
    assert multiply(3, 4) == 12

def test_by_zero():
    assert multiply(9, 0) == 0

def test_by_one():
    assert multiply(7, 1) == 7

def run_tests(tests):
    passed = 0
    for test in tests:
        try:
            test()
            print("PASS", test.__name__)
            passed += 1
        except AssertionError:
            print("FAIL", test.__name__)
    print(f"{passed}/{len(tests)} passed")

run_tests([test_multiplies, test_by_zero, test_by_one])

Changing one character in multiply fixes all of it:

def multiply(a, b):
    return a * b

def test_multiplies():
    assert multiply(3, 4) == 12

def test_by_zero():
    assert multiply(9, 0) == 0

def test_by_one():
    assert multiply(7, 1) == 7

def run_tests(tests):
    passed = 0
    for test in tests:
        try:
            test()
            print("PASS", test.__name__)
            passed += 1
        except AssertionError:
            print("FAIL", test.__name__)
    print(f"{passed}/{len(tests)} passed")

run_tests([test_multiplies, test_by_zero, test_by_one])

Output

PASS test_multiplies
PASS test_by_zero
PASS test_by_one
3/3 passed

The buggy version reports 1/3 passed, and which test survives is the interesting part. multiply(7, 1) returns 8 and fails, multiply(3, 4) returns 7 and fails, but multiply(9, 0) returns 9 while the expected value is 0, so it fails too.

Reading the failures together is what points at the cause. Every wrong answer is the sum of the two arguments, which names the bug precisely without reading the function at all. That habit of looking for the pattern across failures rather than debugging the first one is the core of unit 7.

The tests were not touched during the fix, and that is what makes the green report meaningful.

Surviving a crash, not just a failed assert

Buggy code does not always fail an assert politely, and it can raise any exception at all. A second except clause keeps the runner alive through those too.

def divide(a, b):
    return a / b

def test_divides():
    assert divide(10, 2) == 5.0

def test_by_zero_returns_none():
    assert divide(10, 0) is None

def run_tests(tests):
    passed = 0
    for test in tests:
        try:
            test()
            print("PASS", test.__name__)
            passed += 1
        except AssertionError:
            print("FAIL", test.__name__)
        except Exception as e:
            print(f"FAIL {test.__name__} (crashed: {type(e).__name__})")
    print(f"{passed}/{len(tests)} passed")

run_tests([test_divides, test_by_zero_returns_none])

Output

PASS test_divides
FAIL test_by_zero_returns_none (crashed: ZeroDivisionError)
1/2 passed

The second test documents what the team wants, namely that dividing by zero returns None, while the code raises instead. A crash is a failure too, and the runner has to record it rather than die with it.

Clause order is not optional. except AssertionError comes first and except Exception as e second, because AssertionError is a subclass of Exception and Python takes the first matching clause. Reversing them would swallow every assertion failure into the crash branch and lose the distinction.

type(e).__name__ gives the exception's class name as a string, which is what turns an unhelpful FAIL into a FAIL that names ZeroDivisionError. Lesson 8-2 catalogs the exception types you will see most, and the capstone in unit 10 uses this exact runner.

The exception a failed assert raises

A false assert raises AssertionError, and that is the exact type name, ending in Error and beginning with the name of the statement itself.

The runner catches precisely that type with except AssertionError, records the failure, and moves on to the next test function. You first saw it in lesson 1-1, when a false condition stopped the program.

Knowing the exact name matters for two practical reasons. It is what you write in an except clause, and it is what distinguishes a test that made a wrong prediction from a test whose code broke:

The first row is a failed expectation and the others are crashes, which is why the previous block gave them separate except clauses and separate messages.