Every member action on a professional social network (profile edits, connections, posts, reactions, job applications) is published as an event. A dozen consumers depend on these events, including feed fan-out, search indexing, notifications and analytics. Peak is about 800,000 events a second per data centre, around 1 KB each, across four data centres. Last month a buggy search-indexer deploy corrupted documents for six hours, and nobody could replay the events cleanly.
The interviewer asks“Design the member-activity pipeline on Kafka. Tell me how you guarantee ordering where it matters, how a team replays a day of events after a bad deploy, and what happens when a consumer can't keep up.”
I'd key every event by the entity whose state consumers derive, usually the member ID, and size partitions up front from the slowest consumer's per-partition throughput with two to three years of headroom, because adding partitions remaps keys. Producers run idempotent with acks=all against RF 3 and min.insync.replicas=2. Retention is a day on local disk and 30 days in tiered storage, and replays run as separate rate-limited consumer groups into idempotent, version-checked sinks. Backpressure is explicit at every layer: bounded in-consumer queues with pause and resume, lag-based autoscaling capped at the partition count, producer buffer policies and broker quotas. Schemas are BACKWARD_TRANSITIVE compatible so replays of old data still decode, DLQs park whole keys for order-sensitive consumers, and MirrorMaker 2 runs a local-plus-aggregate topology across DCs.
A free account unlocks the rest of it, and saves your progress across every track.