The feed infrastructure SRE team at a professional social network has six engineers, all in Bengaluru, covering 24/7 with a single person on call for a week at a time. They own about 60 services across several data centres, many of them shared with product teams. Pages average about 140 a week, 45 percent between midnight and 07:00 local time, and the last on-call survey scored stress at 8 out of 10. Two engineers have asked to leave the rotation and one has resigned.
The interviewer asks“You've just taken over an SRE team whose on-call is clearly unsustainable: about 140 pages a week, many of them at night, and two engineers have asked to come off the rotation. How do you bring the load down and design an on-call that people can sustain?”
I'd measure first: pages per shift, pages at night, the share that needed a human, and which alerts produce them, because usually ten alerts cause most of the pain. Then I'd do alert hygiene: delete alerts nobody acts on, turn cause-based ones like CPU into tickets, group and inhibit duplicates, and page on SLO burn rates for member-facing symptoms. In parallel I'd fix the rotation: six people in one timezone on 24/7 is too thin, so I'd pair with a team in another timezone for follow-the-sun twelve-hour shifts, add a secondary, and give time off in lieu after night pages. I'd also cap toil at 50 percent and protect it with a budget. In a past team that took us from 140 pages a week to about 25 in eight weeks, with night pages down by 90 percent.
A free account unlocks the rest of it, and saves your progress across every track.