Concepts
One idea, one trade-off, one widget with two dials.
The fundamentals the problems are built from. Each is a short write-up and a small interactive model built to make a single trade-off visible — not to look like a diagram.
24 concepts in five families
Measuring systems
The vocabulary everything else is written in: what latency and throughput actually are, why a server is full at 70%, what a nine costs, and how to turn 'ten million users' into requests per second.
Latency and Throughput
How long one request takes, and how many you can serve per second — two different axes that people mix up constantly. Little's law is the one equation that ties them together.
Requests in flight = arrival rate × time each one takes. Make anything slower and you need more of everything to hold the same rate.
Little's lawconcurrencyresponse timeTry it →Utilisation and Queueing
Why a server is effectively full long before it reaches 100% busy: waiting time curves upward with utilisation, gently at first and then vertically.
Every point of utilisation you squeeze out costs more latency than the last one. Running at 70% is a choice to buy latency and burst room with a third of the machine.
utilisationheadroomqueueing theoryTry it →Back-of-Envelope Estimation
How to turn 'ten million users' into requests per second, terabytes and gigabits in a minute — the skill every problem in this library opens with.
The numbers are wrong by 2× and that is fine. What matters is which one is big, because that decides the design.
capacity planningQPSstorageTry it →Horizontal vs Vertical Scaling
A bigger machine, or more machines. One is simple and has a ceiling; the other has no ceiling and makes everything else harder.
Scaling up keeps the system simple until the biggest box is not big enough. Scaling out has no top, but now the load must be spread, state must be shared, and any one box can vanish.
scale upscale outredundancyTry it →Availability and the Nines
What 99.9% actually means in minutes, why chaining services multiplies the downtime, and why two copies divide it.
Every dependency in series takes a slice off your availability; every redundant copy in parallel gives one back — at the cost of running it.
99.9%SLAredundancyTry it →Percentiles and Tail Latency
Why the average response time is the wrong number, and why the slowest 1% becomes everyone's problem once a request fans out to many services.
Every extra service a request touches raises the chance it meets a slow one. Ten dependencies at a 1% tail means one request in ten is slow.
p99percentilesfan-outTry it →Stateless vs Stateful
A stateless server remembers nothing between requests, so any copy can serve anyone and copies can come and go. State has to live somewhere — the question is where.
Keeping state on the server is fast and simple until there is more than one server. Moving it out costs a network hop on every request and buys the ability to scale, deploy and fail without anyone noticing.
sessionssticky sessionsshared stateTry it →Idempotency
An operation is idempotent when doing it twice has the same effect as doing it once. It is what makes retries safe — and retries happen whether you planned them or not.
Every retry that reaches a server twice is a duplicate unless the server can recognise it. Recognising it costs a key and a lookup on every write; not recognising it costs a double charge.
idempotency keyretriesexactly-onceTry it →APIs and Communication Styles
How parts of a system talk: request/response over HTTP, streaming over a held connection, or a message dropped on a queue — and when each is the wrong choice.
Polling is simple and wastes requests; pushing is efficient and means holding connections. Synchronous calls are easy to reason about and chain failures; asynchronous ones decouple and complicate.
RESTgRPCWebSocketTry it →CDNs and DNS
The two parts of every request that happen before your servers see it: finding the address, and fetching from a cache near the user instead of from you.
An edge cache turns distance and origin load into a hit-rate problem; a DNS TTL turns 'how fast can I move' against 'how many lookups do I pay for'.
CDNDNSedgeTry it →SQL vs NoSQL
Choosing a data store is choosing what the database will do for you — joins, transactions, a schema — versus what you will do yourself for scale or flexibility.
Normalise and the database keeps every fact once and joins on demand; denormalise and reads are one lookup while every write updates every copy.
relationaldocumentkey-valueTry it →SLOs and Error Budgets
How to decide what 'working' means, measure it, and turn the gap between the target and perfection into a budget you can spend on shipping — or must stop and protect.
A tighter objective means fewer angry users and less room to ship. The budget is the amount of failure you have agreed is fine; burn it fast and you page someone, burn it slowly and you leave it alone.
SLOSLIerror budgetTry it →
Shaping traffic
What happens to requests before they reach the thing that does the work — spreading them, refusing them, and what refused ones do next.
Load Balancing
Spreading requests across a fleet. The strategies only diverge once requests stop costing the same.
Distributing requests evenly is not the same as distributing work evenly.
round robinleast connectionsweightedTry it →Rate Limiting
Deciding whether this request is allowed through, and what a burst is permitted to do.
Every algorithm trades memory per key against how accurately it handles bursts.
token bucketsliding windowburstsTry it →Retries, Backoff and Jitter
A retry turns a blip into a success — and a thousand retries at once turn a blip into an outage. Backoff spreads them out; jitter stops them lining up.
Every retry adds load to the thing that just failed. Backoff and jitter trade a little extra latency for not making the failure worse.
exponential backoffjitterretry stormsTry it →
Placing data
Where a piece of data lives, how many copies, and how a reader finds it fast. Every choice here trades a cheaper read against a dearer write.
Caching
Keeping a copy closer to the reader. The interesting decision is not where reads go but where writes go.
Every write strategy trades durability against how often you touch the store.
cache-asidewrite-throughwrite-backTry it →Indexing
A sorted copy of one column, so a lookup reads a handful of pages instead of the whole table. Every index makes one read cheaper and every write dearer.
Each index you add speeds up the queries that use it and slows down every insert and update that has to maintain it.
B-treecovering indexwrite amplificationTry it →Bloom Filters
A few bits per item that can say "definitely not here" for billions of items from memory. The price is that "maybe" is sometimes wrong.
Fewer bits per item saves memory and raises the false-positive rate; there is no setting where both are free.
probabilisticmembershipfalse positivesTry it →Sharding & Partitioning
Splitting one dataset across many machines, and what the split costs you.
Range partitioning keeps ordering and invites hot spots; hashing kills both.
rangehashhot spotsTry it →Consistent Hashing
Placing keys on nodes so that adding a node moves a small slice of data instead of nearly all of it.
A hash ring moves 1/N of keys when the cluster changes; hash % N moves almost everything.
partitioningvirtual nodesrebalancingTry it →
Decoupling with messages
Putting a buffer between the part that produces work and the part that does it, so neither has to keep the other's pace.
Agreeing under failure
What several machines can promise about the same value when any of them can crash and any message can be late — and what they cannot.
Replication
Keeping more than one copy of the data, and what a failover costs when the copies disagree.
Synchronous replication charges every write; asynchronous replication charges you once, during a failure.
leader-followerfailoverlagTry it →CAP Theorem
What a distributed system does when the network splits in two — the only moment CAP actually applies.
Partitions are not optional, so the real choice is which promise to break when one happens.
partitionsCPAPTry it →Consensus
Getting several machines to agree on one value — a leader, a log entry, a lock — when any of them can crash and messages can be late. A majority is the whole idea.
A majority quorum survives minority failures and never splits, at the cost of a round trip to most of the cluster on every decision.
Raftleader electionquorumTry it →