Idempotency

In plain words

An operation is idempotent if doing it twice has the same effect as doing it once. Pressing a lift button: idempotent. Pressing "buy": not, unless the shop is careful. The word matters because in a distributed system requests will be delivered twice — a reply gets lost, a client retries, a queue redelivers — and whether that is harmless or a double charge depends entirely on whether the operation was designed to be idempotent.

Which operations are already safe

The idempotency key

The client generates a unique id for the intent — "this particular payment" — and sends it with every attempt. The server stores the id with the result. A repeat with a known id gets the stored result and does no work.

app.post("/payments", async (req, res) => {
  const key = req.header("Idempotency-Key");
  if (!key) return res.status(400).send("Idempotency-Key required");

  // Claim the key atomically. If it already exists, someone (maybe us, a
  // moment ago) is handling or has handled this request.
  const claimed = await db.query(
    "INSERT INTO idempotency (key, status) VALUES ($1, 'pending') ON CONFLICT DO NOTHING RETURNING key",
    [key],
  );
  if (claimed.rowCount === 0) {
    const prior = await db.query("SELECT status, response FROM idempotency WHERE key = $1", [key]);
    if (prior.rows[0].status === "done") return res.status(200).json(prior.rows[0].response);
    return res.status(409).send("in progress — retry shortly");   // a concurrent duplicate
  }

  const charge = await cardProvider.charge(req.body.amount, { idempotency_key: key }); // pass it on
  await db.query("UPDATE idempotency SET status = 'done', response = $2 WHERE key = $1", [key, charge]);
  res.status(200).json(charge);
});

Three details that matter:

1. Claim before doing the work

   Insert the key first, atomically. Two retries arriving at once must not both charge; the second must see "pending" and wait.

2. Store the whole response

   The retry should get exactly what the first attempt would have returned — status code and body — so the client cannot tell the difference.

3. Pass the key downstream

   Your payment provider has the same problem with you. Give them the same key (Stripe, Adyen and most others accept one) so your retry to them is safe too.

Where the duplicates come from

• Client retries on a timeout — the case above.
• Queues with at-least-once delivery: a consumer that crashes after the work and before the ack gets the message again. See message queues.
• Gateways and proxies that retry on your behalf, invisibly.
• Users double-clicking, or pressing back and resubmitting.
• Your own retry code at two layers at once — see retries and backoff.

Every one of these is handled by the same key. That is why the key is worth the lookup: it is one mechanism for five failure modes.

Where it goes wrong

• Generating the key on the server. Then every attempt gets a fresh key and nothing is deduplicated. The client must generate it, once, for the intent.
• Keys that live forever. A key table that never expires is a table that grows forever. Keep keys for as long as a retry could plausibly arrive — 24 hours is common — and expire them.
• Reusing a key for a different request. Same key, different amount: the server returns the old result. Some APIs reject a key reused with a different body; do that.
• "Idempotent" but not atomic. Check-then-act without a lock lets two concurrent duplicates both pass the check. The claim must be one atomic operation.

Take this with you

• The one idea: requests will be delivered twice. Make the operation absolute where you can, and give it a client-generated key where you cannot.
• In an interview, every retry, queue and webhook in your design needs a sentence about idempotency. Give it.
• At work, find the POST that moves money or sends a message, and ask what happens when the reply is lost. If the answer is "it happens twice", that is the bug to fix first.