CrackOps
Interview PrepTracksLinuxAWS LabSoonInterview RoomSoon
Sign inStart free
CrackOps

Interview prep for the people who keep production up.

[email protected]
ProductInterview PrepTracksDashboardInterview Room
RolesDevOpsSREPlatformCloud
CompanyAboutContactPrivacyTerms
© CrackOps
Linux / Performance Tuning / pidstat

pidstat

Per-process CPU, memory, disk and context switches over time

Intermediateperformanceprocessessysstat

pidstat reports resource usage per process (or per thread) at a fixed interval: CPU, run-queue wait, memory and page faults, disk read/write rates and context switches. Where top shows a moving snapshot, pidstat gives you a clean, scrollable time series you can paste into a ticket.

Syntaxpidstat [-u] [-r] [-d] [-w] [-t] [-l] [-p PID[,PID...]|ALL] [-C COMM] [INTERVAL [COUNT]]

Watch it run

Load average is high and you want a short per-process CPU history instead of watching top flicker.

devops@devops-lab: ~ — zsh — 96×28
➜ ~
Step 1 of 1
pidstat 2 1

node uses 98% of one CPU. Its %wait of 41% is time it was ready to run but waiting for a CPU: something else competes for CPU 2, or there are more runnable threads than cores. Only processes that were active appear.

Anatomy of a command

$
commandpidstat

Per-process statistics from the sysstat package.

How it works

pidstat reads per-process files under /proc: /proc/PID/stat for CPU ticks, page faults and the CPU last run on; /proc/PID/status and statm for memory; /proc/PID/io for bytes read and written at the block layer; /proc/PID/schedstat for time spent waiting in the run queue (%wait); /proc/PID/status for voluntary and involuntary context switches; and /proc/PID/task/TID for threads with -t. It takes a snapshot, sleeps for the interval, takes another and prints the differences as per-second rates. Without -p it lists only processes that did something in the interval, so a quiet box prints almost nothing. /proc/PID/io of other users' processes is readable only by root, so pidstat -d needs sudo to show them. Percentages are per CPU: a process using 4 cores shows %CPU near 400.

Options you'll actually use

-uCPU: %usr %system %guest %wait %CPU and the CPU number (default)
-rMemory: minor/major page faults per second, VSZ, RSS, %MEM
-dDisk: kB read/written per second, cancelled writes, iodelay (needs root for others)
-wContext switches: voluntary (cswch/s) and involuntary (nvcswch/s)
-tShow threads (TID) as well as processes
-p PID / ALLSpecific processes, comma-separated, or all of them even if idle
-C NAMEOnly processes whose command matches NAME (a regex)
-lShow the full command line instead of the short name
-hOne line per process per interval, no repeated headers: easy to grep

Common mistakes

Running pidstat -d without sudo and concluding nothing does I/O

Other users' /proc/PID/io is root-only, so their rows are silently omitted.

sudo pidstat -d 1
Reading %CPU 380 as 'impossible' or a bug

Percentages are per CPU; a multi-threaded process using almost four cores shows close to 400%.

Compare %CPU with the number of cores, or use -t to see the threads
Watching VSZ to detect a memory leak

VSZ counts reserved address space (thread stacks, mapped files, allocator arenas), which can be huge and harmless.

Watch RSS (and the app's own heap metrics) over time with pidstat -r -p PID 60

Interview questions

Practice

Try these on a Linux machine of your own. Each task has a hint and a solution.

  1. 1

    Show CPU usage of all active processes every second for 5 seconds.

  2. 2

    Watch your own shell's memory (RSS) and page faults, 3 samples, 2 seconds apart.

  3. 3

    Start 'sha256sum /dev/zero &' and find its CPU usage and context switches with pidstat, then stop it.

Related commands

mpstatPer-CPU utilisation: find the hot coresarCollect and replay system activity historytopWatch CPU, memory and processes livepsSnapshot the running processes
PreviousmpstatNextsysctl
Linux / Performance Tuning / pidstat

pidstat

Per-process CPU, memory, disk and context switches over time

Intermediateperformanceprocessessysstat

pidstat reports resource usage per process (or per thread) at a fixed interval: CPU, run-queue wait, memory and page faults, disk read/write rates and context switches. Where top shows a moving snapshot, pidstat gives you a clean, scrollable time series you can paste into a ticket.

Syntaxpidstat [-u] [-r] [-d] [-w] [-t] [-l] [-p PID[,PID...]|ALL] [-C COMM] [INTERVAL [COUNT]]

Watch it run

Load average is high and you want a short per-process CPU history instead of watching top flicker.

devops@devops-lab: ~ — zsh — 96×28
➜ ~
Step 1 of 1
pidstat 2 1

node uses 98% of one CPU. Its %wait of 41% is time it was ready to run but waiting for a CPU: something else competes for CPU 2, or there are more runnable threads than cores. Only processes that were active appear.

Anatomy of a command

$
commandpidstat

Per-process statistics from the sysstat package.

How it works

pidstat reads per-process files under /proc: /proc/PID/stat for CPU ticks, page faults and the CPU last run on; /proc/PID/status and statm for memory; /proc/PID/io for bytes read and written at the block layer; /proc/PID/schedstat for time spent waiting in the run queue (%wait); /proc/PID/status for voluntary and involuntary context switches; and /proc/PID/task/TID for threads with -t. It takes a snapshot, sleeps for the interval, takes another and prints the differences as per-second rates. Without -p it lists only processes that did something in the interval, so a quiet box prints almost nothing. /proc/PID/io of other users' processes is readable only by root, so pidstat -d needs sudo to show them. Percentages are per CPU: a process using 4 cores shows %CPU near 400.

Options you'll actually use

-uCPU: %usr %system %guest %wait %CPU and the CPU number (default)
-rMemory: minor/major page faults per second, VSZ, RSS, %MEM
-dDisk: kB read/written per second, cancelled writes, iodelay (needs root for others)
-wContext switches: voluntary (cswch/s) and involuntary (nvcswch/s)
-tShow threads (TID) as well as processes
-p PID / ALLSpecific processes, comma-separated, or all of them even if idle
-C NAMEOnly processes whose command matches NAME (a regex)
-lShow the full command line instead of the short name
-hOne line per process per interval, no repeated headers: easy to grep

Common mistakes

Running pidstat -d without sudo and concluding nothing does I/O

Other users' /proc/PID/io is root-only, so their rows are silently omitted.

sudo pidstat -d 1
Reading %CPU 380 as 'impossible' or a bug

Percentages are per CPU; a multi-threaded process using almost four cores shows close to 400%.

Compare %CPU with the number of cores, or use -t to see the threads
Watching VSZ to detect a memory leak

VSZ counts reserved address space (thread stacks, mapped files, allocator arenas), which can be huge and harmless.

Watch RSS (and the app's own heap metrics) over time with pidstat -r -p PID 60

Interview questions

Practice

Try these on a Linux machine of your own. Each task has a hint and a solution.

  1. 1

    Show CPU usage of all active processes every second for 5 seconds.

  2. 2

    Watch your own shell's memory (RSS) and page faults, 3 samples, 2 seconds apart.

  3. 3

    Start 'sha256sum /dev/zero &' and find its CPU usage and context switches with pidstat, then stop it.

Related commands

mpstatPer-CPU utilisation: find the hot coresarCollect and replay system activity historytopWatch CPU, memory and processes livepsSnapshot the running processes
PreviousmpstatNextsysctl