A post on X is a few hundred bytes of text, and storing it is the easiest thing the platform does. The trouble starts one step later. That post has to reach everyone who asked to see it, sometimes tens of millions of people, while each of them expects their home timeline to open in a fraction of a second. Nearly every large decision in this design comes from that mismatch: few writes, an enormous number of reads, and a follower graph so skewed that averages describe almost nobody.

This article designs such a platform from the requirements up. It assumes you know what load balancers, caches, and queues do, and spends its time on why they are arranged this way. The order follows A Working Method for System Design, which covers the general process on a smaller problem.

Scope: what has to work first

The feature list for a mature social platform is long, and designing all of it at once produces a diagram with no decisions in it. Sorting it into what defines the product and what can wait gives the design a center.

Direct messages are out of scope. They have different privacy and delivery rules and deserve their own design.

The non-functional requirements are more useful written as decisions than as adjectives:

• Reads stay up when freshness suffers. A timeline ten seconds out of date is fine. One that fails to load is an outage.
• A home timeline returns in under 200 ms at p99, measured at the server. This is our target, and ranking has to fit inside it.
• A post reaches most active followers within five seconds. The number is borrowed from Twitter's stated goal in 2013, a goal the largest accounts made hard to hit.
• An acknowledged post is never lost, even when a machine fails.
• Every stateful component partitions. Nothing may have a capacity ceiling set by one machine.
• Consistency is chosen per feature. Handles must be unique; like counts can drift.
• Every write is rate limited, and every public read is assumed to be scraped. Abuse is a normal workload here.

Rough numbers

Estimates come early because the next decisions depend on them. These are illustrative assumptions, chosen to be round and easy to change. They are not X's traffic.

Follower distribution is the assumption that matters most, and it does not fit in a table cell. Most accounts have a few hundred followers or fewer, and a handful have tens of millions. The 200-insert average leaves out followers inactive for 30 days and leaves out the largest accounts entirely, for reasons the timeline section explains.

The arithmetic, with 86,400 seconds in a day:

The first thing these numbers show runs against the usual summary of social platforms as read heavy. The API is read heavy, with about 30 timeline reads per post, but the timeline cache takes 694,000 inserts per second at peak against 104,000 reads. Precomputing timelines moves the work into a cache that spends most of its effort absorbing writes, and sizing it like an ordinary read-through cache would be a mistake.

Likes outnumber posts twenty to one, so the engagement path needs as much care as the post path even though it looks like a side feature. And 2.6 million post lookups per second rules out hydrating from a database. The post cache has to answer nearly all of them.

For a published reference point, notes on a Twitter talk from 2013 give 400 million tweets a day, 300,000 timeline reads per second, and 30 billion deliveries a day. Those figures are history and say nothing about X today, but they suggest this estimate is a plausible order of magnitude.

The architecture and how requests move through it

This is the design we are working from. It is a sketch for discussion and does not show X's real infrastructure.

Read it as a map of dependencies. An arrow means one box uses another, and it says little about the order things happen in. A few parts are compressed in ways that matter:

• The message queue is drawn behind notifications only. In this design one durable event log sits behind the Post Service, and fanout, search, notifications, and analytics all consume from it.
• The arrow from Posts DB to Fanout does not mean fanout polls the database. It consumes an event emitted when the write commits.
• Ranking sits past the Timeline Cache on the page, but the Timeline Service calls it during a read.

Some labels name real systems. Manhattan, Earlybird, and Heron are a database, a search engine, and a stream processor that Twitter wrote about in 2014, 2012, and 2015. Treat them as historical references. Nothing here claims X runs them today, and any comparable system could fill each slot.

A client request reaches the load balancer, which terminates TLS and spreads traffic across gateway instances. The API gateway checks the token, applies rate limits, and routes the call. It verifies signed tokens locally, because calling the Authentication Service on every request would make that service and the User DB a synchronous dependency of the whole product. Auth is involved at login and token refresh only. Media reads skip this path and go straight to the CDN.

To publish, the gateway forwards the call to the Post Service, which validates it, assigns an ID, and writes to the Posts DB. Once a quorum of replicas confirms, it responds, and an event describing the post is emitted as part of that commit. The author's request is finished at that point. Separate consumers then pick up the event: the Fanout Service reads followers from the Social Graph and fills the Timeline Cache, an indexer updates the Search Index, the Notification Service hands work to its workers through a queue, and the analytics pipeline carries the event through stream processing into a data warehouse for the trends and ML jobs.

To read, the Timeline Service fetches the viewer's entry from the cache, merges in posts from high-follower accounts, calls Ranking for the ranked feed, loads post and author details, applies visibility rules, and returns a page. Nothing on this path touches the Fanout Service, which is the reason to precompute.

The rule I take from this table: a post's success response depends on the Post Service and its storage and on nothing else. Search can be down and fanout can be an hour behind, and posting still works.

Timeline generation

The rest of the design serves this decision. A home timeline is, logically, a query for the newest posts from everyone a person follows, and the question is when that query runs.

Fanout on write

When a user publishes, a worker looks up their followers and inserts the post ID into a precomputed list for each one. Reading then means fetching one list by viewer ID, and its cost no longer depends on how many accounts the viewer follows.

The price is write amplification. One post with 2,000 active followers becomes 2,000 cache inserts, whether or not those followers ever open the app. Twitter's published design from that era worked this way, and the same 2013 notes give useful detail: timelines of up to 800 entries in a Redis cluster, each entry holding a tweet ID, the author's ID, and a few flag bytes, replicated to three machines, with only users active in the last 30 days kept in memory.

Fanout on read

Here nothing happens per follower when a post is written. It lands in the author's own list. When someone opens their timeline, the service loads the accounts they follow, fetches recent posts from each, merges, and sorts.

Writes are cheap and inactive followers cost nothing. Reads get expensive: a viewer following 2,000 accounts triggers 2,000 lookups across many partitions, and the request runs at the speed of the slowest one. Dean and Barroso's The Tail at Scale explains why that fan-out hurts, and we would be paying for it 104,000 times a second at peak.

Why follower count alone is the wrong test

The usual summary is "push for normal users, pull for celebrities". It is roughly right, for a reason that gets skipped. Take one author and one reader. If the author posts p times a day and the reader opens their timeline q times a day, then with a write cost w and a read cost r, pushing costs p × w per day and pulling costs q × r.

Follower count does not appear. The Feeding Frenzy paper (Silberstein et al., SIGMOD 2010) reached this result formally: global cost is minimized by deciding per producer and consumer pair, from the ratio of the producer's update rate to the consumer's view rate. That explains why a follower who has not opened the app in months should never receive inserts, even from a small account, and why a prolific small account can be worth pulling.

Follower count still matters, because of burst size. A single post from an account with 50 million active followers creates 50 million inserts that have to finish within a delivery target. At a spare capacity of 200,000 inserts per second, that one post takes more than four minutes and occupies the whole pipeline. The 2013 notes report this shape of problem, with up to five minutes to reach 31 million followers and replies sometimes arriving in timelines before the post they answered.

So two rules work together. Rate ratios decide who is worth pushing to at all, and burst limits decide which authors cannot be pushed safely.

Hybrid fanout

The design pushes for most authors, skips inactive followers, and treats accounts above a threshold as pull-only. Their posts go to the author's own timeline and stop there. At read time, the Timeline Service merges the viewer's cached list with recent posts from the high-follower accounts they follow.

The threshold needs hysteresis, for example pushing below 50,000 followers and pulling above 100,000, so an account near the line does not flip back and forth. Crossing over requires no migration: older posts stay in caches, and the merge step deduplicates any overlap. It also works as an operational lever. If fanout falls behind, lowering the threshold moves load onto the read path with no deployment.

Merging at read time

The merge is short, and the details are where it goes wrong.

async function homeTimeline(viewer: Viewer, cursor: Cursor, limit = 25) {
  const [cached, pullAuthors] = await Promise.all([
    timelineCache.read(viewer.id, { beforeId: cursor.maxId, count: limit * 2 }),
    graph.highFanoutFollowees(viewer.id, { activeSince: viewer.lastReadAt }),
  ]);

  const pulled = await authorTimelines.recent(pullAuthors, {
    beforeId: cursor.maxId,
    perAuthor: 5,
    deadlineMs: 40, // a late author is dropped, and the page still returns
  });

  return uniqueBy([...cached, ...pulled], entry => entry.canonicalPostId)
    .filter(entry => !viewer.mutedOrBlocked(entry.authorId))
    .sort(byIdDescending)
    .slice(0, limit);
}

Ordering works because Snowflake-style IDs are roughly time-sorted, so entries from both sources compare directly and one maxId cursor pages through both. Deduplication has to use a canonical ID: if two people the viewer follows repost the same post, the entries differ while the content does not.

The pull list needs a ceiling, because a viewer following 3,000 large accounts recreates the fanout-on-read problem for that one request. Passing activeSince limits the lookup to authors who have posted since the viewer's last visit, and the deadline stops one slow partition from holding the page.

Follows and unfollows are handled at the edges. After an unfollow, entries stay in the cache and the authorId stored in each lets the read path filter them without loading the post. After a follow, an asynchronous job backfills the new account's recent posts.

What the public record shows

Twitter's direction in 2013 was to stop fanning out for the largest accounts and merge them at read time, which is where the hybrid design as usually taught comes from.

The published code from 2026 goes further for the For You feed. Its README says the feed is assembled per request, with in-network candidates coming from Thunder, a service that consumes post events from Kafka and keeps recent posts in memory, keyed by author. Each request passes in the viewer's following list. That amounts to a pull across a very fast per-author store, with no per-viewer list on that path.

For a ranked feed this makes sense. Ranking needs a broad candidate set and throws most of it away, so a carefully ordered per-viewer list has less value, and a lookup in local memory avoids the partition fan-out that makes pull slow. The repository covers For You only, so it says nothing about the Following tab or the rest of the product.

I read this as evidence that the choice moves with the workload. When memory is cheap and a read is local, pull gets attractive. When reads are remote and ordering is strict, push does. This article keeps the precomputed cache because it serves a chronological Following timeline well, and it is the clearest place to start.

Data model and storage

Pick each store by how the data is read. Its shape matters far less.

Reposts and replies are rows in the posts table with a kind and a pointer. The 2013 notes describe retweets the same way, as a reference to the original.

CREATE TABLE posts (
  post_id         bigint PRIMARY KEY,  -- Snowflake-style, encodes creation time
  author_id       bigint NOT NULL,
  kind            smallint NOT NULL,   -- post | reply | repost | quote
  body            text,
  repost_of       bigint,              -- repost and quote target
  in_reply_to     bigint,
  conversation_id bigint,              -- ID of the root post in a thread
  media_ids       bigint[],
  deleted_at      timestamptz          -- tombstone, checked at hydration
);

Counters stay off this row. A viral post would turn it into a contention point, so like and repost counts are maintained separately by an aggregator.

Relational or distributed NoSQL

This is rarely an either-or. Accounts want a relational database: the data is small, handles need a uniqueness guarantee, and credential changes benefit from transactions. Posts have the opposite shape, append-heavy, read by key, and huge, which suits a wide-column store built on a log-structured merge tree, where writes are sequential and sharding comes built in.

Be clear about what that costs. Secondary indexes become tables you maintain yourself, which is why the author timeline has its own row above. Cross-partition transactions disappear, and ad hoc queries move to the warehouse. Twitter's Manhattan post describes the same trade: it favored availability and eventual consistency for most use cases and offered strong consistency as an opt-in with explicit costs.

At small scale none of this is needed. One Postgres primary with read replicas will carry a new product for a long time.

The social graph

A graph database sounds like the obvious choice and usually is not, because timeline delivery needs one hop: list this account's followers. FlockDB, Twitter's 2010 graph store, stored graph data without being optimized for traversal. It was built on MySQL for very large adjacency lists, with each edge stored twice, forward and backward, so that "who follows me" and "whom do I follow" each resolve on one partition. Its writes were idempotent and commutative, so operations could repeat or arrive out of order. Facebook's TAO made a similar choice, with a fixed, read-optimized set of queries over objects and associations.

The timeline cache

The cache holds IDs because post text is not needed until hydration, and storing it per follower would multiply memory by the follower count. A Redis list with LPUSH and LTRIM is cheap, but a retried fanout job creates duplicates. A sorted set costs more memory and O(log N) per insert, and gives you uniqueness, tolerance for out-of-order arrival, and range reads for pagination. With at-least-once delivery I would pay that.

There is a trap in the obvious schema. Sorted-set scores are double-precision floats, exact only up to 2⁵³, and a Snowflake ID uses 63 bits. Using the ID as the score silently loses precision and scrambles ordering. Store the ID as the member and the millisecond timestamp as the score; members with equal scores sort lexicographically, so IDs of equal length stay in order. JavaScript numbers have the same limit, which is why API responses carry IDs as strings.

Snowflake-style IDs

Sharded stores have no shared auto-increment, and random UUIDs do not sort. Snowflake was Twitter's 2010 answer: uncoordinated generation of roughly time-sorted IDs in 64 bits. The original format used 41 bits of millisecond timestamp, 10 bits of machine ID, and 12 bits of sequence, or 4,096 IDs per millisecond per machine.

Because of the time prefix, the ID doubles as a sort key: newest-first ordering needs no second column, and a cursor can be a bare ID. The weak point is the clock. The original generator refused to issue IDs if the clock moved backward, and I would keep that behavior, since a brief pause beats a duplicate.

Partitioning and hot keys

Hash the partition key across a fixed number of logical shards, then map those to machines. Moving a logical shard is routine, and rehashing every row is a project. Redis Cluster works this way with 16,384 hash slots.

Hashing spreads keys evenly, but it cannot spread traffic to one key, and here the hot keys are predictable. A celebrity's author timeline is read by every follower at once, so it gets time buckets and a cache. A viral post heats up its row, its count, and its reply partition, so its row is served from an in-process cache with a very short TTL. A huge follower list is paged with cursors so fanout never reads one enormous partition in a single call.

Discord's write-up on storing trillions of messages covers the same problem. Partitions bucketed by channel and time still hit hot spots, and the fix was a service in front of the database that coalesces identical concurrent reads into one query.

Replication and consistency

Posts use quorum replication: with three replicas, a write waits for two. That choice and its limits are covered in Distributed Systems Are About Partial Failure.

Two guarantees need care on top of it. Authors expect read-your-writes even though fanout is asynchronous, and the simplest fix is for the client to insert the post it just created while the server merges the author's own recent posts during reads. A timeline also should not move backward between refreshes, so a session sticks to one cache replica where it can.

API design

The public surface is small. These examples are representative and assume a JSON API behind the gateway.

POST /v1/posts
Authorization: Bearer <access token>
Idempotency-Key: 9f2c1a7e-4b1d-4c8e-9a53-1e0d7a6c2b10
Content-Type: application/json

{ "text": "Shipping the new indexer today", "media_ids": ["1908231044417536"] }

{
  "id": "1908231190374912001",
  "author_id": "42871",
  "text": "Shipping the new indexer today",
  "created_at": "2026-11-01T09:30:12Z"
}

A timeline read returns a page and an opaque cursor:

GET /v1/timelines/home?type=following&limit=25&cursor=eyJ2IjoxLCJtYXgiOiIxOTA4In0

{
  "data": [
    {
      "id": "1908231190374912001",
      "author": { "id": "42871", "handle": "avery" },
      "text": "Shipping the new indexer today",
      "counts": { "likes": 18, "reposts": 2, "replies": 4 }
    }
  ],
  "next_cursor": "eyJ2IjoxLCJtYXgiOiIxOTA3In0",
  "meta": { "ranking": "chronological" }
}

The remaining operations follow the same conventions:

Pagination uses cursors. OFFSET 5000 makes the database read and discard 5,000 rows, and on a feed gaining items at the top, offsets shift and show duplicates. A cursor names a position, so the next page is an indexed range read that costs the same at any depth. FlockDB paged this way in 2010, and Slack's account of moving off offsets is a good read. Keep the cursor opaque and versioned so its contents can change without breaking clients. You lose arbitrary page jumps, which a feed never needed.

Ranked feeds complicate this, because a position in a ranked list means nothing a moment later. There the cursor refers to a session and the server remembers what it already served. The published For You pipeline has a filter for posts already served in the session, which addresses the same concern.

Follow and like are repeatable by design. Creating a post is not, and mobile networks guarantee retries, so the client sends an Idempotency-Key for each logical post and the server returns the stored result on a repeat. The mechanics are in the partial failure article, and Stripe's documentation is a good reference for a production contract.

Mobile apps stay installed for years, so old clients are a permanent condition. Keep most changes additive, as covered in contracts outlive implementations, use a path version like /v1 for real breaks, and announce retirement with a Sunset header.

Media architecture

Media dominates the byte count: 130 TB a day stored against 100 GB of post text. One rule shapes everything here, which is that media bytes never pass through application servers. A video upload through the API would hold a connection and memory on a gateway for minutes.

An upload starts with a call to the Media Service giving the file type and size. The service checks quotas, creates a media_id in state pending, and returns presigned URLs, time-limited credentials that let the holder write one object without any storage credentials of their own. A presigned PUT does not cap the upload size by default, so either use a POST policy with content-length-range or verify the size on completion.

Video uses multipart upload: parts upload separately and a failed part is retried alone, which matters on a phone network. AWS recommends it from 100 MB up. Abandoned uploads keep accruing charges, so a lifecycle rule should clean up incomplete ones.

Nothing the client claimed is trusted. When the upload completes, workers inspect the actual bytes for type and duration, strip location metadata from images, and scan the file, and known illegal media is matched before anything becomes visible. Images then become a few sizes in efficient formats. Video becomes a ladder of bitrates split into short segments for adaptive streaming such as HLS, plus a thumbnail. Each job is keyed by media_id and rendition, so re-running one overwrites identical output. Only when processing finishes does the state move to ready and a post may cite the media.

Renditions get versioned URLs and never change, so they can carry a long max-age with immutable and the CDN serves them without revalidating. At a 95% hit rate, the estimate drops from 880 Gbit/s at the edge to 44 Gbit/s at the origin. Long-tail files move to colder storage classes as they age.

Search architecture

Search works in the opposite direction from the home timeline. As the 2013 notes put it, a timeline does its heavy work on write, while search writes each post to one place and does its work on read.

An inverted index maps each term to a posting list of the documents containing it, so answering redis AND outage means intersecting two lists. Twitter's Earlybird paper describes the real-time variant, where documents must become searchable within seconds while queries run concurrently, and recency dominates ranking, so postings are traversed newest first.

Indexing is asynchronous. The search indexer is one more consumer of the post stream: it tokenizes and normalizes the text and adds it to a shard. Hashtags are ordinary tokens with a prefix, so #redis is a normal term lookup, and author IDs are indexed as fields so filters like from: reuse the same machinery. Documents are partitioned by post ID, which means a query goes to every shard and the Search Service merges the results. Partitioning by time as well helps a great deal, since most queries want recent posts and older shards can cool.

Freshness is a cost you tune. Elasticsearch refreshes every second by default. Twitter reported in 2020 that its indexing latency had been about 15 seconds until mid-2019, and that reaching one second meant removing a sorting buffer and no longer waiting on slow fields such as resolved URLs, which were sent as a later update. Decide what freshness the product needs before paying for it.

The index is derived data, eventually consistent with the Posts DB and always rebuildable from it. A failed batch replays from the log. A deleted post may linger in the index for a while without harm, because search returns IDs and hydration checks the tombstone.

Events and notifications

Everything in the asynchronous half depends on one durable, partitioned log. A Kafka-style log fits because several teams need the same fact, and it retains events after they are read, which allows replay. How Systems Communicate covers the difference between queues and logs.

Getting an event into the log reliably is the first problem. Writing to the database and then to the log can fail between the two, leaving a post that exists but never fans out. A transactional outbox solves this, as does change data capture on the posts table.

The log guarantees order within a partition, so the key decides what is ordered. Keying posts by author keeps one author's posts in sequence, and it also means a very active account makes one partition busy, the hot key problem again.

Each partition is read by exactly one consumer within a group, and each group tracks its own offset. If the search indexer falls two hours behind, fanout does not notice. I think that isolation is the strongest reason to prefer a log over a chain of service calls.

Kafka defaults to at-least-once delivery. A consumer that crashes after doing the work but before committing its offset will have that work repeated. Exactly-once is not worth chasing across external side effects, so every handler has to tolerate a repeat:

A message that always fails would stop an ordered partition forever. After a few quick attempts it moves to a retry topic with backoff, then to a dead-letter queue with an alert and a redrive tool.

A new follower produces one notification. A like might produce a thousand, so engagement notifications collapse on a key such as (recipient, post, type) within a time window, which is how "Avery and 41 others liked your post" works. A worker writes to the recipient's inbox table and then sends a push through the mobile platform services.

The hybrid trick does not help here. Pulling works for a timeline because the reader comes to you, but a push has to go out, so when a large account's subscribers ask for alerts, that fanout really happens. It runs on a separate, rate-limited lane so that it cannot delay mentions and replies.

Ranking and personalization

A chronological timeline is a sort. A ranked one is a recommendation system with its own infrastructure, so keep both: Following stays strictly reverse-chronological and For You is ranked. Ranking can then be the default for users and still be optional for the system, which pays off in the next section.

Ranking usually runs as a funnel, because scoring everything with an expensive model does not fit the time budget:

1. Candidate generation cheaply gathers a few thousand posts, in-network from followed accounts and out-of-network from embedding similarity and popularity.
2. Filtering removes what can never be shown, such as blocked authors and posts already seen.
3. Scoring runs a model over what remains.
4. Re-ranking adjusts the list for qualities a per-post score cannot see, such as variety.
5. Visibility filtering makes the final safety decision for each post and viewer.

The published For You code is the most useful public example of this shape. Its model predicts a probability for many actions, including like, reply, dwell, and follow author, along with negative ones like mute, block, and report. The final score is a weighted sum of those predictions, followed by a decay for repeated authors, a discount for out-of-network posts, and a boost for new authors. A separate service then reorders the list to reduce similarity between neighbors.

Do not mistake that weighted sum for the system. The arithmetic is one line. The hard parts are the model producing the probabilities, the features behind it, the choice of weights, and the experiments that justify them. Any formula in an interview answer is a sketch of the last step.

Inputs come from the viewer's recent actions, which the published system treats as its main model input, along with engagement on the post so far, the relationship between viewer and author, content understanding of text and media, and explicit preferences such as muted words. Serving them within a deadline generally requires a feature store fed by the stream processors, since few can be computed during a request.

A pure engagement objective favors older posts, which have had time to collect signals, and it favors the same few authors. Time limits address the first: the published pipeline drops candidates older than 48 hours. Diversity rules address the second.

Feedback loops are harder. The system learns from what it showed, so new authors get little exposure, generate little data, and stay unseen. Holdout groups, deliberate exploration, and the negative signals above are the usual counterweights. Predicted engagement is also a proxy, and a feed can score well on it while leaving people worse off, so longer-term measures like retention belong in every experiment.

Start with recency, a simple engagement rate, and author affinity. Log impressions from day one, because no later model can be trained without them. A model earns its place once that data exists and the simple version has been measured.

When things fail

For each scenario, the question is what users see and how the system gets back.

A celebrity publishes. The write path is unchanged, since the account is above the threshold and fanout stops at the log. The load moves elsewhere: everyone reads the same post within minutes, so the post cache has to absorb that key, the reply partition heats up, and subscribers get pushes through the rate-limited lane. Timelines stay fast because a read costs one more lookup.

A viral post causes a traffic spike. Autoscaling is too slow for a spike that peaks in minutes, so the protection has to exist already. Identical concurrent reads coalesce into one. Facebook's memcache paper reports that leases, which let one client refill a missing key while others wait briefly, cut peak database load for susceptible keys from 17K to 1.3K queries per second. Counters batch their updates and accept a few seconds of staleness. If load keeps rising, shed it deliberately: drop analytics first, then count freshness, and protect timeline reads last.

Redis becomes unavailable. Losing one node promotes its replica. Replication is asynchronous, so Redis Cluster can lose acknowledged writes and a few entries may vanish, which is acceptable for derived data. Losing a whole shard is the dangerous case, because every affected reader would rebuild from the database at once and could take it down. Requests fall back to a short pull-based timeline behind a circuit breaker, while a rate-limited job rebuilds entries in the background, active users first. Readers get a thinner page and the database survives. The 2013 notes describe the same reconstruction path and give avoiding it as a reason for keeping three copies.

Kafka consumers fall behind. Measure lag in seconds, because a million messages can mean anything. Posts reach timelines late, but posting and reading both still work. Add consumers up to the partition count, and lower the fanout threshold so large accounts stop adding to the backlog. If lag exceeds what a timeline can usefully show, skip the old backlog and let the read path fetch those posts on demand. Log retention must comfortably exceed the longest recovery you expect, or a long outage loses events for good.

Search indexing is delayed. Results go stale and nothing else changes, because the indexer is an isolated consumer. Once fixed, it replays from its committed offset. This delay needs its own alert, since nobody reports a missing result as an error.

A database shard becomes hot. Find the responsible key first, which requires per-key sampling to exist beforehand. A hot key gets caching and coalescing in front of it, while sustained growth means moving logical shards or splitting a time bucket. Resharding during an incident adds risk at the worst moment, so the first response is usually to reduce traffic to that key.

Media processing workers fail. The upload is safe in object storage and the media stays pending. Workers lease jobs, so a crashed job becomes available again when its lease expires, and idempotent rendition keys make the repeat harmless. A file that crashes every worker moves to the dead-letter queue after a bounded number of attempts, and the client is told it failed. Images and video get separate worker pools so a backlog of long transcodes cannot delay a profile photo.

Duplicate events arrive. This is normal operation, and the table in the events section covers it. The one place a duplicate reaches a person is a push notification, so workers check the deterministic ID against a short-lived key before sending.

A timeline cache becomes stale. Cached entries are IDs, so most staleness resolves at hydration: deleted posts hit a tombstone and blocks are checked against current data. The harder case is the post cache, which holds content. A takedown cannot wait for a TTL, so deletes actively invalidate the entry and a short TTL acts as a backstop. The memcache paper's leases also address stale sets, where a slow reader writes an old value back after an invalidation.

The ranking service becomes unavailable. The Timeline Service calls it with a strict deadline, around 100 ms in this design. On timeout, or with a circuit breaker open, it returns candidates in recency order and sets meta.ranking so that dashboards can tell. People get a slightly less interesting feed and may not notice. The fallback is a sort and needs no model, so a failing ranker cannot push its load onto a backup that would also fall over.

Security, moderation, and reliability

Authentication and authorization. Public clients use OAuth 2.0 with PKCE, short-lived access tokens, and rotating refresh tokens. The gateway proves who is calling. Whether this viewer may see this post is decided further in, where protected accounts and blocks are known, so services enforce it themselves.

Rate limiting and DDoS. Limits apply per token, per account, and per IP, with separate budgets for expensive operations such as search. Volumetric attacks are absorbed at the CDN and edge before they reach the load balancer, and the gateway sheds what gets through.

Spam and bots. No single check works for long. Signup friction, device reputation, and behavioral signals feed a rules engine that can react quickly to a new campaign. The published code shows one version: event-driven label rules, account-level models, and reputation scores from PageRank over the follow graph. It also withholds some rules, because publishing them would help people evade them.

Moderation. Ranking and visibility should be separate systems, a split the published code lists as a design decision. Classifiers run as posts appear and write labels to storage. At read time, a visibility service combines those labels with the viewer's relationship to the author and returns one of four outcomes in that system: allow, attach a notice, hide behind an interstitial, or drop. Applying policy at read time means a corrected decision takes effect everywhere at once, with no timeline to rewrite. User reports feed a review queue, and enforcement needs an appeals path with a record, so that automation does not reapply an overturned action.

Privacy. A post exists in many derived copies: caches, search, the log, and the warehouse. Each needs a defined deletion path and deadline, and the right to erasure makes that a legal requirement in some jurisdictions. Events should carry IDs and minimal data, and warehouse access should be restricted and audited.

Observability. Alert on what users experience. The most useful single metric here is time from post creation to arrival in a timeline, sampled with synthetic accounts, because it exercises the whole asynchronous path. Track timeline p99, consumer lag in seconds, cache hit rate, ranker fallback rate, and indexing delay alongside it. Trace IDs should travel through event headers as well as RPC calls. Google's Dapper paper established the sampled tracing model, and Zipkin began at Twitter as an implementation of it. Page on SLO burn rate, which catches fast failures and slow leaks with fewer false alarms than fixed thresholds.

Disaster recovery. Accounts, posts, the graph, and media are sources of truth and need cross-region replication, backups, and tested restores. Caches, indexes, and trends can be rebuilt, though the rebuild has to be rate limited and rehearsed. Treat a backup that has never been restored as a hypothesis.

The final design, and what to build first

The services divide along differences in load and failure, which is a better guide than the nouns in the product. Posting needs durability, timeline reads need latency, and fanout needs throughput with no one waiting, so those three should not share a scaling curve or a failure domain. Where no such difference exists, a separate service only adds a network hop, a point argued at length in Microservices Are an Ownership Decision.

Consistency lines up with that division:

Asynchronous processing removes work from the request path wherever nobody is waiting. Caching is essential in two places, timeline IDs and hydrated posts, because nothing else on the read path could reach the required rates. Everywhere else a cache is an optimization.

Almost none of this should be built on the first day. A sensible progression:

1. Start with a modular monolith and Postgres, with timelines built by fanout on read through an indexed query. Put media in object storage behind a CDN immediately, since that is cheap now and painful to change later.
2. Add Redis and background jobs when timeline queries become the slowest thing in the system, and move to fanout on write through a simple job queue.
3. Add search, notifications, and the event log as those features earn real usage, with each new consumer subscribing to the log.
4. Introduce the hybrid threshold when delivery latency for your largest accounts shows up in measurements.
5. Partition storage when a single primary reaches its limit, and add a ranked feed once impression data can support one.

A few decisions are expensive to change later and belong in the earliest version: time-sortable IDs, cursor pagination, idempotency keys, string IDs in the API, and an event for every state change. System Architecture Is the Set of Decisions That Are Expensive to Change has more on telling those apart from the ones you can safely revisit. The rest of the diagram can wait until a measurement asks for it.

Further reading

• Raffi Krikorian, Timelines at Scale (InfoQ, 2013), and the High Scalability notes cited throughout.
• Adam Silberstein et al., Feeding Frenzy: Selectively Materializing Users' Event Feeds (SIGMOD 2010).
• Michael Busch et al., Earlybird: Real-Time Search at Twitter (ICDE 2012).
• Nathan Bronson et al., TAO: Facebook's Distributed Data Store for the Social Graph (USENIX ATC 2013).
• Rajesh Nishtala et al., Scaling Memcache at Facebook (NSDI 2013).
• X, x-algorithm: the open-source For You feed, for a current look at candidate generation, ranking, and visibility filtering.