News Feed

In plain words

A news feed shows you the latest posts from the people you follow. Computing that on every refresh means combining hundreds of lists, and people refresh far more often than they post — so the design does the combining once, when a post is made, instead of every time someone looks. Then it deals with the one case where that is impossible: an account with twenty million followers.

The numbers

Assume 10 M daily users opening the feed 30 times a day and posting 3 times.

feed reads, average — 30 K at peak

3,500 /s

posts

350 /s

followers, median user

200

followers, the biggest accounts

20 M

Worked the way back-of-envelope estimation describes: a few facts, a few multiplications, and the big number tells you the shape of the problem.

The asymmetry is the design. Three hundred writes a second can afford to be expensive. Thirty thousand reads a second cannot.

Decision 1: fan out on write

When a user posts, a worker reads their follower list and appends the post id to each follower's timeline — a per-user list in Redis, capped at a few hundred entries. A feed read is then one list fetch plus a batched hydration (fetching the full post for each id). No join, no sort, no query across followed accounts.

// On post: push the id onto every follower's timeline.
async function fanOut(postId: string, authorId: string) {
  const followers = await followerCache.get(authorId);        // changes slowly; cache it
  const pipe = redis.pipeline();
  for (const f of followers) {
    pipe.lpush(`timeline:${f}`, postId);
    pipe.ltrim(`timeline:${f}`, 0, 799);                       // keep the newest 800
  }
  await pipe.exec();
}

// On read: one list, one batched hydrate.
async function feed(userId: string) {
  const ids = await redis.lrange(`timeline:${userId}`, 0, 49);
  return postStore.getMany(ids);                                // one multi-get, not 50 queries
}

The price is paid at write time: a post by someone with 200 followers is 200 list appends. At 350 posts/s that is 70,000 appends/s, which a small Redis fleet handles without noticing. Turn the followers per post slider up in the model and watch where it stops being small.

Decision 2: the celebrity problem

Fan-out on write breaks the moment one account has enough followers. A post from an account with 20 M followers is 20 M appends — minutes of work for the worker pool, during which every other post queues behind it. A burst of celebrity posts is an outage.

async function feed(userId: string) {
  const [timelineIds, celebs] = await Promise.all([
    redis.lrange(`timeline:${userId}`, 0, 49),
    followsCache.celebritiesFollowedBy(userId),               // usually 0–20 accounts
  ]);
  const celebIds = (await Promise.all(celebs.map((c) => postStore.recentIds(c, 20)))).flat();
  const merged = mergeByTime(timelineIds, celebIds).slice(0, 50);
  return postStore.getMany(merged);
}

Reads pay a small fixed cost (a few extra lookups) for a large variable one that writes could not pay at all. Every real feed uses some version of this.

Decision 3: keep media off the path

Images and video are most of the bytes and none of the logic. They go straight from the client to object storage on upload and straight from the CDN (a content delivery network: caches placed close to users) to the client on view; the feed carries URLs. A feed page with twenty images is one API call and twenty CDN hits, and the origin behind the CDN — your own storage — sees only the misses.

Where this design breaks

• Timeline cache misses. An evicted (dropped to make room) timeline is rebuilt with a query per followed account — the naive read path, twenty times more expensive. Keep the hit rate high or the store takes the full read load on a cold restart.
• Ranking. The design above orders by time. A ranked feed scores candidates at read time, which brings compute back to the hot path — which is why ranked feeds precompute scores in the fan-out step.
• Follower list reads. The worker reads the poster's follower list for every post. Cache it; it changes slowly and is read constantly.
• Deletes and edits. A deleted post is already in a million timelines. Filter at read time rather than trying to un-fan-out.

Take this with you

• The one idea: move work from the busy path (reads) to the quiet path (writes) — and switch back to read-time merging for the few accounts where the write-time cost explodes.
• In an interview, say "fan-out on write, hybrid for celebrities", give the numbers, and point out that the timeline is a cache, not the truth.
• At work, measure your timeline hit rate. A cold cache turns every read into the expensive path you designed away.