What breaks first

The bottleneck

Traffic grows. At some point one resource on the box hits its limit, and every request queues behind it. That resource is the bottleneck: the narrowest point in the pipe. Fixing anything other than the bottleneck changes nothing, which is why finding it comes before fixing it.

On a single server, the usual suspects in the order they tend to fail:

1. Database time. Tables grow, queries that scanned 1,000 rows now scan 10 million
2. CPU. The processor is busy 100% of the time, requests wait for a turn
3. Memory (RAM). The app or database needs more working space than exists
4. Connections. The server can only hold so many open network connections

In practice the database is the first and most common bottleneck, and units 3 through 5 are devoted to it.

A capacity check

This takes the average RPS from lesson 1-1, applies a 10x peak factor, and compares the result against what the box can handle.

avg_rps = 0.93
peak_rps = avg_rps * 10
capacity_rps = 8
print("Peak requests per second:", round(peak_rps, 1))
if peak_rps > capacity_rps:
    print("Overloaded: requests queue up and time out")
else:
    print("The server keeps up")

Output

Peak requests per second: 9.3
Overloaded: requests queue up and time out

The whole calculation is one multiplication and one comparison, and that is the point. Capacity planning is not sophisticated math, it is remembering to do the multiplication before the busy hour rather than after.

Note what happens past the limit, since it is worse than slow. Requests do not politely wait their turn, they pile up in a queue, and each one gets slower until clients start timing out and retrying, which adds load to an already overloaded box.

capacity_rps = 8 is a stand-in for something you would measure rather than guess. Unit 8 covers how to get that number from real traffic, and until then treat it as a placeholder that happens to be roughly right for a modest box with a database on it.

Raising capacity_rps to 12 would give the healthy case, where capacity sits above peak load. Healthy also means having headroom, so a system running at 99% of capacity is one small traffic bump away from the overloaded branch.

Where the time goes

To find a bottleneck, measure where each request spends its time. A typical breakdown for one request on our box:

The database takes 80% of the time. If the server handles requests one at a time, the maximum throughput is 1000 ms ÷ 25 ms = 40 requests per second. Real servers overlap requests, but the ratio still tells you what to fix first: making app code twice as fast saves 5 ms, making the query twice as fast saves 10 ms.

This measuring habit has a name you will meet again in unit 8: observability.

Turning a timing breakdown into numbers

The same breakdown as the table, computed rather than read off.

app_ms = 5
db_ms = 20
total_ms = app_ms + db_ms
print("Time per request (ms):", total_ms)
print("Database share:", str(round(db_ms / total_ms * 100)) + "%")
print("Max requests per second:", 1000 // total_ms)

Output

Time per request (ms): 25
Database share: 80%
Max requests per second: 40

The share calculation is round(db_ms / total_ms * 100) with a % concatenated on using str(), since Python will not add a string to a number. Rounding to a whole percent is the right precision for a figure whose inputs are measured in whole milliseconds.

1000 // total_ms uses integer division because a fractional request per second is not a useful answer. Forty is the sequential ceiling, meaning what one worker handling one request at a time could do.

Real servers beat that number by overlapping requests, and the reason is that most of the 25 ms is spent waiting. While one request waits on the database, the server can work on another, which is the same event-loop insight from the backend course.

What the ratio gives you is the direction to optimize, not the absolute limit. Eighty percent of the time is in the database, so that is where any real gain has to come from.

Note how quickly this shifts if the numbers change. A query that grows to 200 ms makes the database 97% of the request, and at that point the app code could take zero time and barely help.

What happens when you optimize the app code

Little changes, because the database was the bottleneck and still takes 20 ms.

You optimized the part that was not the bottleneck. Total time drops from 25 ms to 21 ms, which is a 16% gain rather than the 5x speedup the app-code improvement might suggest.

The general form of this is worth naming, since it comes up constantly. Speeding up a component that accounts for a fraction of the total can only ever save that fraction, so a 5x improvement on 20% of the work is a 16% improvement overall.

The lesson that drives this whole course is to identify the bottleneck first and then spend effort there. Effort spent elsewhere is not merely wasted, it is worse than that, because it consumes the time and attention the real problem needed.

Right now the bottleneck is the database, so that is where we go after learning to scale the app tier. Unit 2 handles the app side first because it is the easier half, and units 3 through 5 are the database.

Peak load against capacity

Peak is 4 x 8 = 32 requests per second, which is above the 30 RPS capacity.

The site is fine for 23 hours a day and falls over during the busy hour. That pattern is what makes this failure mode so common, since every dashboard averaged over a day says the system is healthy.

Peak equals average times peak factor, and that one line of arithmetic is the entire capacity check. Skipping it is how a launch goes badly on the first evening.

Being 2 RPS over the limit is also worth a moment. Overload is not gradual, because once arrivals exceed service rate the queue grows for as long as the overload lasts, so 32 against 30 is not a 7% slowdown, it is an hour of climbing latency and timeouts.

The fixes, adding capacity or spreading load, are exactly what unit 2 covers. There are only two directions to go, which is a bigger box or more boxes, and each one has a different set of consequences.