Agreeing under failure
ConsistencyCAP Theorem
What a distributed system does when the network splits in two — the only moment CAP actually applies.
Partitions are not optional, so the real choice is which promise to break when one happens.
Try it
Move the dials — the sentence under the picture changes.In plain words
Your data lives on several machines. One day the network between them breaks — a partition — and each side can only see itself. Now a machine gets a request. It can answer with what it has (which might be stale, or might conflict with what the other side is doing), or it can refuse until it can check with the others. That is the whole of the CAP theorem: during a partition, pick available or consistent. You cannot have both.
What the theorem actually says
"Pick two of three" is a misleading summary. Partition tolerance is not a choice — if your system runs on more than one machine, the network will split eventually. So there is one question, not three:
And it only applies during a partition. The widget makes this concrete: with the link intact, both modes behave identically. Cut the link and they finally differ.
CP: refuse rather than be wrong
A CP system needs a quorum — a majority of the copies — to agree before it answers. On the smaller side of a partition, reads and writes fail. Nothing is ever stale, nothing diverges, and the users on that side see an error.
- Account balances and payments
- Inventory you will actually ship
- Configuration that controls other systems
- Leader election, locks
etcd, ZooKeeper, Spanner, a single Postgres primary.
- Shopping carts (merge them later)
- Timelines, feeds, likes
- Presence, sessions, metrics
- Every cache
Dynamo-style stores, Cassandra at low quorums, DNS.
AP: answer with what you have
An AP system keeps serving on both sides. Reads may be stale; writes accepted on both sides produce two versions of the truth that must be reconciled when the link returns. Choosing AP is choosing to write a merge rule.
Keep the version with the later timestamp. Easy — and it silently throws away a real write whenever two machines' clocks disagree, which they do.
A version counter per replica, so the store can tell "newer" from "written at the same time". Concurrent versions are both kept and handed to the application to merge. Now every reader needs a merge rule.
Data types that merge by construction — a counter that adds, a set that unions — so any order of updates ends the same. Perfect for carts and counters; not general.
// Both sides accepted writes during the partition. Merge = union of items,
// quantities summed; a removal wins only if it is newer than the add.
function mergeCarts(a: Cart, b: Cart): Cart {
const items = new Map<string, Item>();
for (const it of [...a.items, ...b.items]) {
const cur = items.get(it.sku);
if (!cur) items.set(it.sku, it);
else items.set(it.sku, { ...it, qty: cur.qty + it.qty, removedAt: latest(cur.removedAt, it.removedAt) });
}
return { items: [...items.values()].filter((i) => !i.removedAt || i.removedAt < i.addedAt) };
}The trade you make every day: PACELC
CAP describes a partition, which is rare. PACELC adds the part you live with all the time: Else, when the system is healthy, you are still trading Latency against Consistency. A read that checks a quorum is slower than a read from the nearest copy — whether or not anything is broken.
Ask a majority of replicas, take the newest. Always fresh. 3 round trips, maybe across regions: 80 ms.
Ask the nearest replica. Might be a second stale. 1 ms.
In practice this second trade shapes far more of your system than the first one ever will. Most reads in most systems are the local kind, on purpose.
Take this with you
- The one idea: partitions are not optional. The only choice is what to do during one: refuse, or answer without being sure.
- In an interview, do not say "pick two". Say which data is CP and which is AP in your design, and why — and mention the everyday latency trade (PACELC).
- At work, most of your data is AP already (caches, sessions, feeds). Make sure the few things that must be CP — money, inventory, config — actually are.