The iteration protocol

What a for loop really does

This unit looks under the hood of for, and the payoff is threefold: it explains a whole family of real-world bugs (loops that mysteriously run empty), it lets your own classes work with for, and it is the foundation of generators, the tool professionals use to process data too large to hold in memory.

You have looped over lists, strings, and dicts. Why does for work on all of them? Because they all follow one contract, the iteration protocol:

1. iter(thing) asks for an iterator, an object that remembers a position.
2. next(iterator) returns the next value.
3. When the values run out, next raises the special StopIteration exception.

So this loop:

for x in [10, 20]:
    print(x)

is really:

it = iter([10, 20])
while True:
    try:
        x = next(it)
    except StopIteration:
        break
    print(x)

Anything that can hand out an iterator is called an iterable.

Iterators are one-shot

An iterator moves forward only. Once exhausted, it stays empty, you must call iter again for a fresh pass. This explains a bug you will absolutely hit someday: looping twice over something that is an iterator and finding the second loop does nothing.

A concrete case. zip(a, b) is a built-in that pairs up two sequences item by item, first with first, second with second. It returns an iterator over those pairs, not a list:

pairs = zip([1, 2], ["a", "b"])
print(list(pairs))   # [(1, 'a'), (2, 'b')]
print(list(pairs))   # [] , already used up

Open files behave the same way: in unit 8 you will see that a file object hands out its lines exactly once per opening.

Lists are iterables but not iterators: each for loop asks them for a brand-new iterator, so you can loop over a list as many times as you like.