Caching

Keeping a copy closer to the reader. The interesting decision is not where reads go but where writes go.

Every write strategy trades durability against how often you touch the store.

cache-asidewrite-throughwrite-backhit rate

Try it

Move the dials — the sentence under the picture changes.

Write strategy

Reads80% reads

Cache-aside invalidates on write, so the next read for that key pays a miss. Simple and safe, and it is why the hit rate sits at 0% rather than higher — writes keep knocking entries out.

In plain words

A cache is a small, fast copy of data kept close to whoever keeps asking for it. Your browser caches images so a page you revisit loads instantly; a service caches database rows so a popular product page does not hit the database ten thousand times a minute. The bet is simple: the same things get asked for again and again, and in real systems that bet almost always pays.

Reads go to the cache first. Only the ones it cannot answer — the misses — reach the database.

Reading is the easy half

On a read the app asks the cache. A hit returns in a fraction of a millisecond. A miss means going to the database, then putting the answer in the cache for next time. The share of reads that hit is the hit rate, and it is the number that decides how much the cache is worth.

Writing is where the decisions are

The moment data changes, the copy in the cache is wrong. What you do about that is the real choice, and there are three answers.

Cache-aside: the app talks to both. Read from the cache, fall back to the database and fill; on a write, update the database and delete the cached copy.

Cache-aside

default

The app owns the logic: read cache → miss → read DB → fill. Write DB → delete the cache entry.

Cache outage degrades to slow, not wrong.
Only caches what is actually read.
Every write costs the next reader one miss.

Write-through

Every write goes to the cache and the database together, before the caller gets an OK.

Cache is never stale.
Every write pays twice.
Caches things nobody reads.

Write-back

careful

Write only to the cache; flush to the database later, in batches.

Absorbs write bursts; fewest DB writes.
A cache crash loses acknowledged writes.
Right for counters and metrics, wrong for orders.

Cache-aside, the way most services write itTypeScript

async function getProduct(id: number): Promise<Product> {
  const key = `product:${id}`;
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);          // hit

  const row = await db.query("SELECT * FROM products WHERE id = $1", [id]);
  await redis.set(key, JSON.stringify(row), { EX: 60 }); // fill, 60 s TTL
  return row;
}

async function updatePrice(id: number, price: number) {
  await db.query("UPDATE products SET price = $1 WHERE id = $2", [price, id]);
  await redis.del(`product:${id}`);               // invalidate, do not update
}

Making room: eviction

A cache that never throws anything away is just a slower database. When it is full, something has to go.

Policy	Drops	Good at	Weak spot
LRU — least recently used	What nobody has touched for longest	Most workloads	One scan of cold data (a report, a crawler) evicts everything valuable
LFU — least frequently used	What has been read least often	Stable popularity	Clings to what was popular yesterday
TTL — time to live	Anything older than N seconds	Bounding staleness	Not really eviction — it is a promise about freshness, used with the others

Keeping it fresh: invalidation

People joke that cache invalidation is one of the two hard problems in computer science. It is hard because the cache does not know the data changed unless someone tells it, and "someone" is spread across every code path that writes.

Start with a TTL only
Accept that data can be up to N seconds old. For a product price, 60 s is usually fine. Most caches should stop here.
Add explicit deletes for the things that must be fresh
Stock count, account balance. Delete the key on every write path — and know that a missed path means stale data.
Or change the key instead of the value
product:42:v7. Bump the version on write; the old key simply ages out. No invalidation call to lose.

Some data sidesteps the problem entirely. A URL shortener's code → URL mapping never changes once written, so it can be cached forever.

Where it goes wrong

Thundering herd

A hot key expires at 12:00:00. In the next 50 ms a thousand requests all miss and all hit the database for the same row. The database, which was fine, falls over.

Fix

Let one request refill (a short lock on the key) while the rest wait or serve the stale value. Or refresh hot keys a little before they expire.

Cache penetration. Requests for keys that do not exist miss every time and go straight through — an attacker can pick ids that are never there. Cache the negative answer (product:999 → NOT_FOUND, short TTL).
Cold start. A restarted cache fleet sends 100% of reads at the database. Restart nodes one at a time, or warm the cache from a list of hot keys.
Hot keys. Sharding a cache spreads keys evenly, not traffic. One viral product still lands on one node. Replicate the key, or add a small in-process cache in front for the top hundred.

Take this with you

The one idea: a cache is a bet on repetition. Reads are easy; decide on purpose what a write does to the copy.
In an interview, say cache-aside with a TTL unless there is a reason not to, and name the thundering herd on a hot key expiring.
At work, find your hit rate and your worst key. A 95% hit rate means the database still takes 5% of reads — and 100% of them on a cold restart.