The traffic director
A load balancer (LB) is a server whose only job is to receive every incoming request and forward it to one of your app servers. The browser talks to a single address, the load balancer's, and never knows how many servers sit behind it.
What a load balancer gives you:
- Distribution. Spreads requests so no single server is overwhelmed
- Health checks. It pings each server every few seconds. A server that stops answering is removed from rotation, and users never see its errors
- Elasticity. Add or remove servers at any time. The LB just updates its list
How does it pick a server? The two algorithms to know:
- Round robin: take turns. Server A, B, C, A, B, C. Simple and fair when requests are similar
- Least connections: send each request to the server currently handling the fewest. Better when some requests are much slower than others
Code exercise · python
Run this round-robin simulation. The modulo operator % cycles through the server list: request i goes to server i % 3.
Code exercise · python
Your turn: implement least connections. The dict holds how many active requests each server has. For each of 5 incoming requests, pick the server with the fewest active requests (min with a key function), add 1 to it, and print the assignment. Then print the final load dict.
Quiz
Server B behind your load balancer crashes. What do users experience, assuming health checks are configured?
Who balances the load balancer?
One worry should nag you: the load balancer is a single machine sitting in front of everything. Did we just move the single point of failure from lesson 2-1 instead of removing it?
Yes, and the fix is standard practice. A load balancer does almost no work per request (no app code, no database queries), so one machine can forward enormous traffic, and you rarely need many. For failure you run a redundant pair: an active load balancer plus a standby that monitors it with the same heartbeat idea as health checks, and takes over its network address within seconds if it dies. Very large systems add DNS-level balancing above that, publishing several load balancer addresses so browsers spread across them.
The habit to take away: for every tier you draw, ask what happens when this one machine dies. Interviewers ask exactly that, and this course will keep answering it tier by tier.
Quiz
Your only load balancer fails at 3 am. In a production setup, what limits the damage?