Rate Limiting

Why servers say "slow down"

Without limits, one client can hammer your API: a password-guessing bot on /login (the attack lesson 7-4 warned about), a buggy script in a retry loop, or a scraper pulling your whole database. Each request costs you CPU, database load, and money.

A rate limiter counts requests per client (usually per IP address or per user) inside a time window and rejects the excess with status 429 Too Many Requests (from the lesson 3-3 table).

The simplest scheme is the fixed window: allow at most N requests per minute, reset the counts when the next minute starts. The core is just a Map of counters, which is exactly what you will build, minus the clock so the output stays deterministic.

Counting requests per client

allow(ip) reads the current count for that ip, returns false when the limit is used up, and otherwise increments and returns true.

function createLimiter(limit) {
  const counts = new Map();
  return {
    allow(ip) {
      const used = counts.get(ip) || 0;
      if (used >= limit) return false;
      counts.set(ip, used + 1);
      return true;
    },
  };
}

const limiter = createLimiter(3);
for (let i = 1; i <= 5; i++) {
  console.log("req " + i + ": " + (limiter.allow("1.2.3.4") ? "200" : "429"));
}
console.log("other ip: " + (limiter.allow("9.9.9.9") ? "200" : "429"));

Output

req 1: 200
req 2: 200
req 3: 200
req 4: 429
req 5: 429
other ip: 200

counts.get(ip) is undefined for a first-time ip, so || 0 normalizes it and the first request of every client takes the same path as the rest. Without that fallback the arithmetic would produce NaN and the comparison would silently allow everything.

Check used >= limit before incrementing, or you allow one extra request. With > instead of >= the limiter of 3 would pass four, which is the classic off-by-one in this function.

Each ip has its own counter, which is why the last line still says 200. A noisy client is isolated to its own entry, and that is the entire purpose, since a global counter would punish everyone for one bad actor.

counts lives in the closure returned by createLimiter, so nothing outside can reach in and edit it. That is the same encapsulation the repository used in lesson 6-2, and it means limit is fixed at creation time.

Note that this is a fixed window without the clock, so the counts never reset yet. Adding time is the next step, and it is the only thing standing between this and a usable limiter.

Resetting the window

A fixed window means the counters are wiped when a new time window starts, and reset does exactly that with counts.clear.

function createLimiter(limit) {
  const counts = new Map();
  return {
    allow(ip) {
      const used = counts.get(ip) || 0;
      if (used >= limit) return false;
      counts.set(ip, used + 1);
      return true;
    },
    reset() {
      counts.clear();
    },
  };
}

const limiter = createLimiter(2);
console.log(limiter.allow("1.2.3.4"));
console.log(limiter.allow("1.2.3.4"));
console.log(limiter.allow("1.2.3.4"));
limiter.reset();
console.log(limiter.allow("1.2.3.4"));

Output

true
true
false
true

counts.clear() is the whole body of reset, and clearing the map is cheaper and safer than deleting keys one at a time. It also drops entries for clients that have gone away, which keeps memory from growing with every ip that ever visited.

The four output lines are one window ending and another beginning. The blocked client is welcome again immediately after the reset, without any change on its side.

In production the reset is driven by the clock, with setInterval(() => limiter.reset(), 60000) starting a fresh window every minute. Keeping the clock outside the limiter is what makes the tests above deterministic, which is the injectable-dependency idea from the testing unit.

The fixed window has a known weakness called the boundary burst. A client can send its full limit at 11:59:59 and its full limit again at 12:00:00, so it gets twice the intended rate across a two-second span.

Fixed window is still the right first thing to build, because it stops the bots and buggy retry loops that motivate limiting in the first place. Reach for token bucket when the burst behavior actually matters.

What happens when you scale to three servers

Each server counts separately, so a client effectively gets about three times the limit.

Each process has its own memory, so each server sees roughly a third of the traffic and enforces the limit against its own private counters. Nothing is broken in the code, the assumption that one process sees all requests simply stopped holding.

This is the same lesson the in-memory repository taught in unit 6, arriving from a different direction. Any state kept in process memory is per-instance state, and horizontal scaling is what exposes that.

Production systems put the counters somewhere shared, typically Redis, a small and very fast data store that all your servers reach over the network, so every server increments the same counters. Redis has atomic increment commands built for exactly this, which also avoids two servers reading the same value at once.

In-memory limiting still helps as a first line of defense, and it is worth keeping even alongside a shared store. It costs no network round trip, and it survives a Redis outage, so a common design is a local limiter with a generous ceiling in front of a shared one with the real limit.

Real limiters also add the Retry-After header to their 429 responses, telling the caller how many seconds to wait. A well-behaved client then backs off correctly instead of retrying immediately and making the situation worse.