mpstatPer-CPU utilisation: find the hot core
mpstat breaks CPU time down per core: user, system, iowait, interrupts, softirqs, steal and idle. It reveals what a single 'CPU 25%' number hides: one core pinned at 100% by a single-threaded process, interrupts all landing on CPU 0, or the hypervisor stealing your cycles.
mpstat [-P {N[,M...]|ALL}] [-u] [-I {SUM|CPU|SCPU|ALL}] [-n] [INTERVAL [COUNT]]Dashboards show the 8-core box at 16% CPU, yet one job is crawling. You want to see each core.
mpstatMultiprocessor statistics, from the sysstat package.
mpstat reads /proc/stat, where the kernel keeps per-CPU counters of time spent in each mode since boot (in clock ticks): user, nice, system, idle, iowait, irq, softirq, steal, guest, guest_nice. With an interval it takes two snapshots and turns the differences into percentages of that window, for each CPU and for all CPUs together. Without an interval it prints a single report averaged since boot, which is rarely what you want. -I reads /proc/interrupts and /proc/softirqs to show interrupt rates per CPU. %steal is time the hypervisor ran something else while this virtual CPU wanted to run; %guest is time this machine spent running its own VMs. iowait is a kind of idle time, not work: it only means the CPU had nothing to run while I/O was outstanding.
-P ALLOne row per CPU plus the average (-P 0,2 for specific CPUs)INTERVAL COUNTSample every INTERVAL seconds, COUNT times, then print averages-uCPU utilisation report (the default)-I SUM / CPU / SCPUInterrupts per second: total, per hardware IRQ, per softirq type-nPer-NUMA-node summary instead of per CPU-o JSONJSON output for scripts%usr / %sysTime in user space / in the kernel on behalf of processes%soft / %irqTime handling softirqs (network, timers) / hardware interrupts%stealTime the hypervisor didn't give this vCPU (cloud: throttling)Concluding there's no CPU problem because overall CPU is 20%On 8 cores, one core at 100% is only 12.5% of the total. Single-threaded work hits its ceiling long before the average looks busy.
mpstat -P ALL 1 to see each coreRunning mpstat with no interval and treating it as current loadWithout an interval it reports averages since boot.
mpstat 1 5 or mpstat -P ALL 2Treating high %iowait as CPU saturation and adding coresiowait is idle time waiting for disk. More CPUs won't make storage faster.
Check the disks: iostat -x 1 or sar -d -p 1Ignoring %steal on cloud VMsSustained steal means the instance isn't getting its CPU (burst credits exhausted or a busy host); tuning the app won't fix it.
Check CPU credit metrics, move to an unlimited or non-burstable instance typeTry these on a Linux machine of your own. Each task has a hint and a solution.
Show utilisation for every CPU, sampled once per second, 3 times.
Start a busy loop with 'yes > /dev/null &' and find which core it pins, then stop it.
Show how many interrupts per second the system handles in total.
mpstatPer-CPU utilisation: find the hot core
mpstat breaks CPU time down per core: user, system, iowait, interrupts, softirqs, steal and idle. It reveals what a single 'CPU 25%' number hides: one core pinned at 100% by a single-threaded process, interrupts all landing on CPU 0, or the hypervisor stealing your cycles.
mpstat [-P {N[,M...]|ALL}] [-u] [-I {SUM|CPU|SCPU|ALL}] [-n] [INTERVAL [COUNT]]Dashboards show the 8-core box at 16% CPU, yet one job is crawling. You want to see each core.
mpstatMultiprocessor statistics, from the sysstat package.
mpstat reads /proc/stat, where the kernel keeps per-CPU counters of time spent in each mode since boot (in clock ticks): user, nice, system, idle, iowait, irq, softirq, steal, guest, guest_nice. With an interval it takes two snapshots and turns the differences into percentages of that window, for each CPU and for all CPUs together. Without an interval it prints a single report averaged since boot, which is rarely what you want. -I reads /proc/interrupts and /proc/softirqs to show interrupt rates per CPU. %steal is time the hypervisor ran something else while this virtual CPU wanted to run; %guest is time this machine spent running its own VMs. iowait is a kind of idle time, not work: it only means the CPU had nothing to run while I/O was outstanding.
-P ALLOne row per CPU plus the average (-P 0,2 for specific CPUs)INTERVAL COUNTSample every INTERVAL seconds, COUNT times, then print averages-uCPU utilisation report (the default)-I SUM / CPU / SCPUInterrupts per second: total, per hardware IRQ, per softirq type-nPer-NUMA-node summary instead of per CPU-o JSONJSON output for scripts%usr / %sysTime in user space / in the kernel on behalf of processes%soft / %irqTime handling softirqs (network, timers) / hardware interrupts%stealTime the hypervisor didn't give this vCPU (cloud: throttling)Concluding there's no CPU problem because overall CPU is 20%On 8 cores, one core at 100% is only 12.5% of the total. Single-threaded work hits its ceiling long before the average looks busy.
mpstat -P ALL 1 to see each coreRunning mpstat with no interval and treating it as current loadWithout an interval it reports averages since boot.
mpstat 1 5 or mpstat -P ALL 2Treating high %iowait as CPU saturation and adding coresiowait is idle time waiting for disk. More CPUs won't make storage faster.
Check the disks: iostat -x 1 or sar -d -p 1Ignoring %steal on cloud VMsSustained steal means the instance isn't getting its CPU (burst credits exhausted or a busy host); tuning the app won't fix it.
Check CPU credit metrics, move to an unlimited or non-burstable instance typeTry these on a Linux machine of your own. Each task has a hint and a solution.
Show utilisation for every CPU, sampled once per second, 3 times.
Start a busy loop with 'yes > /dev/null &' and find which core it pins, then stop it.
Show how many interrupts per second the system handles in total.