Idempotency keys0/59 done
Part 3 · System design · Reliability

Idempotency keys

Lets a client safely retry a request that changes something, such as a payment, without doing it twice.

How it works

A timeout is ambiguous: the server may or may not have done the work. So the client sends a unique key per operation in an Idempotency-Key header. The server claims the key before doing anything (a UNIQUE row, or Redis SET NX). The same key with the same body gets the stored response replayed, and the work does not run again. The same key with a different body is a client bug: reject it. If the first request is still running, answer 409 so the client tries again shortly. Keys expire after a day or so.

first requestclaim the key, do the work, store the response
retry, same bodyreplay the stored response
retry, different body422
retry while the first is running409, try again

Reach for it when

Implementation (JavaScript, tested)
// Same key + same body: replay the stored response. Same key + different body: reject.
// Production: a table with a UNIQUE key in the same database as the side effect (or Redis SET NX + TTL).
class IdempotencyStore {
  constructor() {
    this.m = new Map();
  }
  handle(key, body, run) {
    const fp = JSON.stringify(body);
    const prev = this.m.get(key);
    if (prev) {
      if (prev.fp !== fp) return { status: 422, body: "key reused with a different request" };
      if (prev.res === null) return { status: 409, body: "still processing, retry later" };
      return prev.res; // replay; the work does not run again
    }
    this.m.set(key, { fp, res: null }); // claim the key BEFORE doing the work
    const res = run();
    this.m.get(key).res = res;
    return res;
  }
}

Traps

In real systems: Stripe's Idempotency-Key header is the reference design. AWS APIs take a client token. Kafka's idempotent producer uses a producer id plus a sequence number.
Say out loud: Timeouts are ambiguous, so writes take an idempotency key. I claim it first, in the same database as the side effect, and replay the stored response on retries.