Retries, Backoff and Jitter

A retry turns a blip into a success — and a thousand retries at once turn a blip into an outage. Backoff spreads them out; jitter stops them lining up.

Every retry adds load to the thing that just failed. Backoff and jitter trade a little extra latency for not making the failure worse.

exponential backoffjitterretry stormsidempotency

Try it

Move the dials — the sentence under the picture changes.

Retry policy

Burst1500 clients

Capacity600/s

Max attempts6

Everything that fails comes straight back, so 6 attempts are used up in 0.3 s and 1,320 of 1500 requests give up — the service never got a chance to drain the burst.

In plain words

When a request fails, trying again usually works — most failures are a blip: a dropped packet, a server mid-restart, a lock held a moment too long. So every client library retries by default. The trouble starts when the failure was not a blip but an overload: now every failed request comes back, the service that could not cope gets more traffic, more requests fail, more come back. That is a retry storm, and retries caused it.

Retries add traffic to the thing that is already too busy. Without backoff, a 10% overload becomes a 300% one.

Backoff: wait, and wait longer each time

The first fix is to pause before retrying, and to double the pause each time: 100 ms, 200, 400, 800, 1,600. This is exponential backoff. A client that has failed four times is now retrying about once a second instead of ten times a second, so the retry load on a struggling service falls away quickly instead of piling up.

Two guard rails go with it: a cap on the wait (say 30 s) so it does not grow forever, and a limit on attempts (say 5) so a permanent failure is not retried until the heat death of the universe.

Jitter: spread the wave

Randomise the wait. Instead of exactly 400 ms, wait a random amount between 0 and 400 ms ("full jitter"). A thousand retries are now spread across the whole window instead of landing together; the service sees a gentle slope instead of a cliff.

Same number of retries either way. Jitter changes when they land.

Exponential backoff with full jitterTypeScript

async function withRetry<T>(fn: () => Promise<T>, attempts = 5): Promise<T> {
  const base = 100, cap = ;                       // ms
  for (let i = 0; ; i++) {
    try {
      return await fn();
    } catch (err) {
      if (i + 1 >= attempts || !isRetryable(err)) throw err;
      const ceiling = Math.min(cap, base * 2 ** i);      // 100, 200, 400, 800…
      const wait = Math.random() * ceiling;              // full jitter: 0..ceiling
      await sleep(wait);
    }
  }
}

function isRetryable(err: unknown): boolean {
  const status = (err as { status?: number }).status;
  if (status === 429 || status === 503) return true;    // "I'm busy" — back off
  if (status && status >= 400 && status < 500) return false; // our fault — will fail again
  return true;                                           // timeout, connection reset
}

The same idea applies anywhere many clients act on the same timer: reconnecting after a gateway restarts, refreshing a cache entry that expired at the same second, cron jobs on the minute. Jitter is the answer to all of them.

Only retry what is safe

A retry after a timeout does not know whether the first attempt happened. The request may have reached the server and been processed; only the reply was lost.

Safe to retry

GET /orders/42. Running it twice returns the same thing twice. Retry freely.

Not safe — unless…

POST /payments "charge $50". Run it twice and the customer pays $100 — unless the server can recognise the second attempt as the first.

The fix is an idempotency key: the client generates a unique id for the operation and sends it with every attempt. The server remembers keys it has seen and returns the original result for a repeat instead of doing the work again.

With an idempotency key, a retry is harmless: the server answers from memory instead of charging again.

Never retry an error that says the request itself was wrong — a 400, a validation failure. It will be wrong again.

Where it goes wrong

Retries at every layer. The browser retries 3 times, the gateway 3 times, the service 3 times: one failure becomes 27 attempts. Retry at one layer — usually the outermost — and pass failures through elsewhere.
Retrying on "I'm busy". A 503 or 429 is the service asking for less. Retrying quickly is the storm. Honour Retry-After, and back off harder than for a timeout.
No budget. Cap retries as a share of traffic (10% is common); beyond that, fail fast. A circuit breaker — stop calling a failing dependency for a while, then send one probe — is this idea with memory.
Deadlines that do not shrink. Five retries of a 10 s timeout is 50 s of the user staring at a spinner. Give the whole operation one deadline and stop when it is spent.

Take this with you

The one idea: retries turn an overload into a storm unless they back off and are jittered. Backoff sets how much retry traffic; jitter sets its shape.
In an interview, say "exponential backoff with full jitter, a cap, an attempt limit, and only for idempotent requests".
At work, count the retry layers between a user and your database. If it is more than one, a single failure is multiplying.