Off-by-One Hunting

Boundaries: where off-by-one bugs live

An off-by-one error is a bug where code is wrong by exactly one unit, almost always at a boundary: the value where behavior is supposed to change. Free shipping at 50. Grade A at 90. Teen from 13 to 19. The classic cause is mixing up > and >=, or < and <=.

The hunting technique is called boundary value analysis and it is mechanical:

1. find each boundary value in the spec
2. test exactly at the boundary
3. test one step below and one step above

Three cheap tests per boundary. A > where >= belongs passes the below and above tests but fails the at test, so this trio catches the entire bug family.

The boundary trio in practice

Free shipping starts at 50, so the three probes are 49.99, 50, and 50.01. All three pass because the code correctly uses >=.

def free_shipping(total):
    return total >= 50

def test_just_below_pays():
    assert free_shipping(49.99) == False

def test_exactly_at_boundary_is_free():
    assert free_shipping(50) == True

def test_just_above_is_free():
    assert free_shipping(50.01) == True

for test in [test_just_below_pays, test_exactly_at_boundary_is_free, test_just_above_is_free]:
    test()
    print("PASS", test.__name__)

Output

PASS test_just_below_pays
PASS test_exactly_at_boundary_is_free
PASS test_just_above_is_free

The middle test is the one that earns the trio its place. A mistaken total > 50 would still pass the 49.99 and 50.01 tests, so without a probe exactly at the boundary the bug ships. That is the entire argument for the technique in one example.

What counts as one step depends on the data. Here the totals are money, so 49.99 and 50.01 are the neighboring values, while for an integer age the steps would be 17 and 19. Choosing a step of 1 on a float boundary, testing 49 and 51, would leave the whole range between 49 and 50 unprobed.

Note that the assertions compare against True and False explicitly. The function returns the result of a comparison, so assert free_shipping(50) would work too, and spelling out the expected value makes the below and above cases read symmetrically.

Finding a wrong comparison at the A boundary

The spec says 90 and above is an A, but a student with exactly 90 was graded B. Before the fix, letter_grade used score > 90 on its first line.

def letter_grade(score):
    if score >= 90:
        return "A"
    if score >= 80:
        return "B"
    return "C"

def test_exactly_90_is_A():
    assert letter_grade(90) == "A"

def test_89_is_B():
    assert letter_grade(89) == "B"

def test_exactly_80_is_B():
    assert letter_grade(80) == "B"

def test_79_is_C():
    assert letter_grade(79) == "C"

passed = 0
tests = [test_exactly_90_is_A, test_89_is_B, test_exactly_80_is_B, test_79_is_C]
for test in tests:
    try:
        test()
        print("PASS", test.__name__)
        passed += 1
    except AssertionError:
        print("FAIL", test.__name__)
print(f"{passed}/{len(tests)} passed")

Output

PASS test_exactly_90_is_A
PASS test_89_is_B
PASS test_exactly_80_is_B
PASS test_79_is_C
4/4 passed

With the bug in place, the report reads 3/4 passed and only test_exactly_90_is_A fails, which points straight at the line deciding the A grade. score > 90 is false when the score is exactly 90, so the value falls through to the >= 80 line and comes back as a B.

The fix is one character, changing > to >= on the A line. The B line already used >= correctly, which is a useful detail, since inconsistency between two adjacent lines is itself a smell worth noticing while reading.

This is also the first example in the course where the runner's try and except from lesson 1-3 does real work. The failing test is the first one in the list, so a plain loop would have crashed before reporting on the other three, and you would not have learned that the B boundary was fine.

The same hunt on an inclusive range

The spec says a teen is 13 to 19 inclusive, and someone who is exactly 13 was told they are not a teen. Before the fix, the lower bound read 13 < age.

def is_teen(age):
    return 13 <= age <= 19

def test_12_not_teen():
    assert is_teen(12) == False

def test_exactly_13_is_teen():
    assert is_teen(13) == True

def test_exactly_19_is_teen():
    assert is_teen(19) == True

def test_20_not_teen():
    assert is_teen(20) == False

passed = 0
tests = [test_12_not_teen, test_exactly_13_is_teen, test_exactly_19_is_teen, test_20_not_teen]
for test in tests:
    try:
        test()
        print("PASS", test.__name__)
        passed += 1
    except AssertionError:
        print("FAIL", test.__name__)
print(f"{passed}/{len(tests)} passed")

Output

PASS test_12_not_teen
PASS test_exactly_13_is_teen
PASS test_exactly_19_is_teen
PASS test_20_not_teen
4/4 passed

Only test_exactly_13_is_teen fails with the bug present, which identifies the lower bound as the culprit without reading the rest of the line. 13 < age is false when age is exactly 13, and the spec says inclusive, so <= is what belongs there.

A range with two boundaries needs four probes rather than six, and that is worth understanding. The trio for each boundary would give 12, 13, 14, 18, 19, and 20, but the just-inside values of 14 and 18 add nothing that 13 and 19 have not already established, so the practical set is one outside and one at each end.

Python's chained comparison, 13 <= age <= 19, reads like the spec and evaluates as two comparisons joined by and. Writing it as two separate conditions is equally correct, and the chained form is easier to check against the written requirement, which is exactly the kind of small readability choice that prevents this bug in the first place.

Choosing a boundary test set

For a password valid at lengths 8 to 20 inclusive, the best test set is lengths 7, 8, 20, and 21.

There are two boundaries, so the probes are one step outside and exactly at each of them. 7 must be rejected, 8 accepted, 20 accepted, and 21 rejected.

Each of the four probes rules out a specific bug, which is why none of them is redundant:

A set of only 8 and 20 misses both rejections, so a validator that accepted everything would pass all of it. That is the failure mode to watch for in any test set, since a suite that only ever asserts success cannot distinguish working code from code that always says yes.

A set like 10 and 15 never touches the boundaries at all and so cannot detect an off-by-one anywhere. Middle-of-the-range values confirm the feature exists, and boundary values are what confirm it is correct.