Where caches live

Which step took 80% of each request

The database query, at 20 of the 25 ms.

Unit 2 scaled the app tier, and every app server still hammers the same database. Ten app servers behind a load balancer means ten times the query load on one database, so scaling the app tier can make the real bottleneck worse.

Caching attacks that 20 ms directly, by answering repeated questions from memory instead of asking the database again. It is the first tool in this course that reduces work rather than adding capacity to absorb it.

Note which lever this is in lesson 1-3's terms. Adding servers multiplied throughput and left latency alone, and a cache does the opposite, cutting the per-request time and freeing the database as a side effect.

What a cache is

A cache is a small, fast store that keeps copies of recently used data so repeated requests skip the slow, authoritative source. The database stays the source of truth. The cache is a disposable copy: losing it costs speed, never correctness.

The payoff is enormous because real traffic repeats itself. A read from Redis (the shared in-memory store from lesson 2-3) takes well under 1 ms. The same read from the database takes 10 to 50 ms.

Two words you will use daily:

• Cache hit: the data was in the cache, answer instantly
• Cache miss: it was not, fetch from the source, usually storing a copy for next time

The fraction of requests that hit is the hit rate, the single number that tells you whether a cache is earning its keep.

The four layers where caches live

A request from a browser can be answered from four caches, each one closer to the database:

A CDN (content delivery network) is a rented fleet of caching servers spread around the world, so a user in Tokyo gets your logo from Tokyo instead of your data center in Virginia. You met the latency reason in How the Internet Works: distance costs milliseconds, roughly 100 ms for a trans-Pacific round trip.

Rule of thumb: serve each request from the layer closest to the user that can correctly answer it.

Which layer fixes slow photos in Australia

A CDN that caches the photos on servers near Australian users.

The problem is distance, not database load. The photos are static files that the database is not even involved in serving, so no amount of query optimization or Redis capacity touches this.

Redis in the US is still a US round trip away from Sydney, which is the key realization. A cache only helps if it is on the right side of the slow part, and here the slow part is 15,000 km of fiber.

A CDN puts copies of static files physically near users, cutting the trip from roughly 200 ms to roughly 20 ms. Images are the ideal CDN content, since they are large, identical for every user, and change rarely.

Each cache layer fixes a different bottleneck, so name the bottleneck first, which is lesson 1-2 again in a new setting. The symptom here contains the diagnosis, because fast in one region and slow in another is a geography problem by definition.

How much load a 90% hit rate removes

Only misses touch the database, so 20,000 x 0.10 = 2,000 reads per second.

The cache absorbed 90% of the load, which is a tenfold reduction, and the database went from a number most single instances cannot serve to one that is comfortable.

Notice the sensitivity, because it is not linear in the way people expect. At a 99% hit rate the database sees just 200 reads per second, so improving the hit rate from 90% to 99% removes another 90% of the remaining load.

Read that table in the other direction for the more useful lesson. A hit rate falling from 99% to 90% multiplies database load by ten, which is why a small regression in cache behavior can look like a database outage.

Small hit-rate improvements produce huge database savings, which is why engineers watch hit rate like a vital sign. It belongs on the dashboard next to latency and error rate, and a sudden drop is an early warning rather than a curiosity.