An Indian UPI payments app processes 9,000 payments per second on a sale day. One large issuer bank, about 14 percent of volume, starts timing out on 40 percent of debit requests. Pay API threads are piling up waiting on those requests, p99 latency for all banks has doubled, and the in-doubt transaction count is rising.
The interviewer asks“One issuer bank is timing out on a large share of debits during a sale, and it's starting to hurt everyone else. How do you decide whether to degrade or disable that bank, or a whole payment method, and how do you bring it back?”
I decide with thresholds agreed before the day, not during the incident: technical decline rate and latency per bank and per flow, with a minimum volume, measured against how much that bank is hurting everyone else. There's a ladder: first isolate the bank with its own concurrency limit and tighter timeout so it can't starve shared threads, then nudge users to pay from another account, then block new debits to that bank with a clear message. The on-call IC can flip pre-approved per-bank switches immediately; disabling a whole rail like UPI collect or a card network needs a named business owner. Re-enable is a ramp, 5, 25, 50, 100 percent, each step gated on success rate and latency, while in-doubt transactions are resolved by status checks throughout.
A free account unlocks the rest of it, and saves your progress across every track.