← Concepts

Measuring systems

Foundations

CDNs and DNS

The two parts of every request that happen before your servers see it: finding the address, and fetching from a cache near the user instead of from you.

An edge cache turns distance and origin load into a hit-rate problem; a DNS TTL turns 'how fast can I move' against 'how many lookups do I pay for'.

CDNDNSedgeTTLorigin

Try it

Move the dials — the sentence under the picture changes.
User is
CDN
Userother continentEdge95% hits · 12 msOrigin140 ms awayPage load (DNS + 30 assets)125 msRequests reaching the origin, per page view1.5 of 30
At 95% edge hits, the page loads in about 125 ms and the origin sees 1.5 requests per page view instead of 30. DNS TTL 300 s: moving the site to a new address takes up to that long to reach everyone.

In plain words

Before a request reaches any server you run, two things happen. DNS turns the name in the address bar into an IP address — the phone book. Then, for anything static — images, scripts, video segments — a CDN (content delivery network) answers from a cache in a data centre near the user instead of from your origin on the other side of the world. Together they decide how far a request travels and how much of your traffic you actually serve yourself.

once per TTLevery assetmisses onlyUserSingaporeDNSname → IP, cachedCDN edgeSingapore · 12 msOriginVirginia · 140 ms
The name is resolved once and cached. The bytes come from the edge; only misses cross the ocean.

DNS: the lookup and the TTL

Resolving systemdesign101.online walks a chain — the browser's cache, the operating system's, the ISP's resolver, then the root, .online, and finally your authoritative server — and each level caches the answer for the record's TTL (time to live). A 30 ms lookup you pay once per TTL, not per request.

A DNS record, and what its TTL means
systemdesign101.online.   300   IN   A       203.0.113.10
;                         ^^^
;                         seconds a resolver may keep this answer

api.systemdesign101.online.  60  IN  CNAME   lb-.example-cloud.net.
;                            ^^
;                            short: this one moves during failover
Long TTL (hours)

Fewer lookups, less load on your DNS, slightly faster first requests. But changing the address — a failover, a migration — takes hours to reach everyone, and some resolvers hold on longer than you asked.

Short TTL (seconds–minutes)

You can move the site in a minute. Every client looks you up more often, and if your DNS provider is down, so are you, quickly.

DNS is also where geo-routing happens: the same name answers with the nearest edge's address, which is how a CDN sends Singapore to Singapore. And it is a single point of failure people forget to draw — the availability of your DNS provider is a term in yours.

CDN: a cache with a map

A CDN is caching with geography added: hundreds of edge locations, each a cache, each answering the users nearest to it. The rules are the cache rules — hit rate is the number, invalidation is the hard part — plus one about distance.

user to the nearest edge
12 ms
Singapore to Virginia, one way and back
140 ms
typical edge hit rate for static assets
95–99%
of requests your origin actually serves
1–5%

What to put on it

Immutable assets
cache forever

JS and CSS bundles with a content hash in the name (app.7f3a9c.js), images, fonts, video segments. Never change once written, so max-age can be a year. This is most of the bytes.

Semi-static pages
short TTL

A product page, a news article. Cache for a minute; accept a minute of staleness; purge on edit if you must be exact.

Personalised responses
do not

"Your cart", "your feed". Different per user; a cache hit here is a privacy bug. Cache-Control: private, and let it pass through.

Headers that make the edge do the right thingHTTP
# The bundle: content-addressed, so it can be cached by everyone forever
Cache-Control: public, max-age=31536000, immutable

# A product page: a minute at the edge, always revalidate in the browser
Cache-Control: public, s-maxage=60, max-age=0, stale-while-revalidate=300

# The users cart: never shared
Cache-Control: private, no-store

s-maxage is the edge's TTL, max-age the browser's; stale-while-revalidate lets the edge serve the old copy while it fetches a new one, so users never wait for a refresh.

Where the origin still hurts

  • Cold cache, hot content. A new video goes viral: the first thousand viewers in every region all miss at once. Good CDNs collapse identical simultaneous requests into one origin fetch; make sure yours does.
  • The long tail. 5% misses on a huge catalogue is still a lot of requests, all for things nobody else wants. Size the origin for the misses, not for zero.
  • Purging. "Invalidate everything" is a stampede on the origin. Purge by key, and prefer changing the URL (a new hash) to purging at all.
  • Cache-busting query strings. ?v=2 on a file the edge caches by full URL means every variant is a separate miss. Fine when intended, a hit-rate disaster when a tracking parameter is added to every link.

Take this with you

  • The one idea: distance is latency you cannot code away; a CDN moves the bytes next to the user, and DNS is the switch that points them there.
  • In an interview, put static assets behind a CDN with hashed names, state the hit rate, and set a short TTL on anything that has to move.
  • At work, check what share of your traffic reaches the origin. If it is more than a few percent for static content, look at the headers.