A live-sports streaming platform runs its playback API on Kubernetes across three regional clusters. Fifteen minutes before a football final, viewers start joining; the Grafana CPU panel for the playback pods reads 92%, and p99 latency is climbing. The HorizontalPodAutoscaler targets 70% CPU with minReplicas 40 and maxReplicas 200, but the Deployment has sat at 40 replicas for twelve minutes.
The interviewer asks“Your dashboard says the pods are above 90% CPU, the HPA target is 70%, and the replica count hasn't moved. How do you find out why the HPA isn't scaling, and what do you change so it can't happen at the next toss?”
I'd stop trusting the dashboard and read the HPA's own view with kubectl describe hpa: the current utilisation it computed, desired versus current replicas, and the three conditions. HPA CPU utilisation is usage over requests, averaged over every container in every ready pod, so a sidecar with a big request, a container with no request, unready pods, or a dashboard showing percent of limit can all explain the gap. If the HPA did ask for more pods, I follow the chain downstream: maxReplicas, behaviour policies, ResourceQuota on the ReplicaSet, Pending pods waiting for nodes, or GitOps resetting replicas. Right now I'd raise minReplicas to get capacity, then fix the metric — container-level targets or a request-rate metric — and pre-scale before scheduled matches.
A free account unlocks the rest of it, and saves your progress across every track.
kubectl get List Kubernetes resources and their statuskubectl describe Explain an object's state, conditions and eventskubectl logs Read and follow container logskubectl exec Run a command or shell inside a running containerkubectl apply Create or update resources from YAML, declarativelykubectl delete Delete Kubernetes resources safelykubectl scale Change the replica count of a workloadkubectl rollout Watch, undo and restart Deployment rollouts