Failure spreads through calls
Lesson 8-1 warned that microservices turn function calls into network calls. Here is the concrete disaster that follows, the one behind many famous outages. The recommendations service gets slow, taking 30 seconds instead of 20 ms. The product page service calls it and waits. Waiting requests hold threads and connections, so the product page service runs out of capacity, and the home page service calling it starts waiting too. One sick service has taken down three healthy ones. That is a cascading failure, and the root cause was not the slowness, it was the unlimited waiting.
Three defenses, applied at every network call between services:
- Timeout. Never wait longer than a budget, like 200 ms. A slow dependency becomes a fast, explicit error instead of a held thread
- Fallback. When the call fails, degrade instead of erroring the whole page: render the product page without the recommendations strip. Most users never notice, and the page stays up
- Circuit breaker. After several consecutive failures, stop calling the sick service at all for a while and go straight to the fallback. That sheds load off the struggling service so it can actually recover, the same logic that made lesson 6-2 back off its retries. Periodically one trial call is let through, and if it succeeds the circuit closes again
Code exercise · python
Run this circuit breaker simulation. After 3 consecutive failures the circuit opens: later calls skip the sick service entirely and serve the fallback immediately.
Code exercise · python
Your turn: compute a worst-case latency budget. A request passes through a chain of 3 services. Each hop has a 2-second timeout and performs 1 retry after a timeout. Print the worst case per hop and for the whole request, exactly as shown.
Budgets compound
Twelve seconds of worst case from three innocent 2-second timeouts is the lesson: timeout budgets compound along call chains. Deep chains therefore need small per-hop budgets, very few retries, and ideally an overall deadline passed down the chain, with each service subtracting the time already spent before calling the next. It is also one more quiet argument for lesson 8-1's honesty about microservices: every service boundary you add is another place where timeouts, fallbacks, and breakers must be designed, tested, and monitored.
In an interview, saying "every cross-service call gets a timeout, a fallback, and a circuit breaker" the moment you draw your first service-to-service arrow is exactly the failure story lesson 9-2 says to volunteer before being asked.
Quiz
The payments service is completely down. Which behavior shows a well-designed checkout service calling it?