CrackOps
Interview PrepTracksLinuxAWS LabSoonInterview RoomSoon
Sign inStart free
CrackOps

Interview prep for the people who keep production up.

[email protected]
ProductInterview PrepTracksDashboardInterview Room
RolesDevOpsSREPlatformCloud
CompanyAboutContactPrivacyTerms
© CrackOps
Linux / System Information / vmstat

vmstat

Report CPU, memory, swap and I/O activity over time

Intermediateperformancememorymonitoring

vmstat prints one compact line per interval: run queue, blocked tasks, memory, swap in/out, block I/O, interrupts, context switches and the CPU split. In a few seconds it tells you whether a slow box is short of CPU, memory or disk.

Syntaxvmstat [-S M] [-w] [-t] [DELAY [COUNT]] vmstat -s | -d

Watch it run

The API is slow. Check, second by second, whether the CPUs are the bottleneck.

devops@devops-lab: ~ — zsh — 96×28
➜ ~
Step 1 of 1
vmstat 1 5

Ignore line 3: it's the average since boot. From line 4 on, r (runnable tasks) is 4 to 6 on a 4-CPU box, us is around 90 and wa is 0. The CPUs are saturated by user code; disk and memory are fine.

Anatomy of a command

$
commandvmstat

procps-ng vmstat: reads /proc/stat, /proc/meminfo and /proc/vmstat each interval.

How it works

vmstat reads kernel counters and prints the differences between samples. procs r and b come from /proc/stat (procs_running and procs_blocked: runnable tasks and tasks in uninterruptible sleep). Memory comes from /proc/meminfo. si/so (KiB/s swapped in from and out to disk) and bi/bo (blocks per second read from and written to block devices, 1 KiB units) come from /proc/vmstat counters. in and cs are interrupts and context switches per second; us, sy, id, wa, st and gu are percentages of CPU time, from the tick counters in /proc/stat. The first line is special: it is averaged since boot, so ignore it when diagnosing a current problem. vmstat costs almost nothing to run, which makes it safe on a box that is already struggling.

Options you'll actually use

DELAY [COUNT]Print every DELAY seconds, COUNT times (first line = since boot)
-S MMemory columns in MiB (k, K, m, M allowed)
-wWide output: wider columns that don't overflow
-tTimestamp on every line
-aShow active/inactive memory instead of buff/cache
-sOne-off table of memory and event counters
-dPer-disk read/write statistics
-ySkip the since-boot first line

Common mistakes

Reading the first line of vmstat 1 as the current state

The first line is an average since boot, often calm, while the problem is happening now.

Skip it, or use vmstat -y 1
Panicking at a non-zero swpd

swpd is how much is sitting in swap, possibly idle pages from days ago. That alone doesn't slow anything.

Watch si/so: sustained swap traffic is what hurts
Expecting bi/bo in megabytes

bi and bo are blocks per second, 1 KiB blocks on Linux. 180000 means about 180 MB/s.

Divide by 1024 for MiB/s, or use iostat -m
Comparing r with 1 instead of with the CPU count

r is the number of runnable tasks; a 16-CPU box can run 16 at once without queueing.

Sustained r above nproc means CPU saturation

Interview questions

Practice

Try these on a Linux machine of your own. Each task has a hint and a solution.

  1. 1

    Print 5 vmstat samples, one per second, with memory in MiB.

  2. 2

    Run vmstat for 3 samples without the since-boot line and with timestamps.

  3. 3

    Show the one-off summary of memory and event counters, and find how many forks happened since boot.

  4. 4

    Generate CPU load with `yes > /dev/null &` and watch r and us change in vmstat. Then stop it.

Related commands

iostatMeasure disk throughput, latency and utilisationfreeShow used, free and available memorytopWatch CPU, memory and processes livesarCollect and replay system activity historyuptimeHow long the system has run, and its load average
PreviousuptimeNextiostat
Linux / System Information / vmstat

vmstat

Report CPU, memory, swap and I/O activity over time

Intermediateperformancememorymonitoring

vmstat prints one compact line per interval: run queue, blocked tasks, memory, swap in/out, block I/O, interrupts, context switches and the CPU split. In a few seconds it tells you whether a slow box is short of CPU, memory or disk.

Syntaxvmstat [-S M] [-w] [-t] [DELAY [COUNT]] vmstat -s | -d

Watch it run

The API is slow. Check, second by second, whether the CPUs are the bottleneck.

devops@devops-lab: ~ — zsh — 96×28
➜ ~
Step 1 of 1
vmstat 1 5

Ignore line 3: it's the average since boot. From line 4 on, r (runnable tasks) is 4 to 6 on a 4-CPU box, us is around 90 and wa is 0. The CPUs are saturated by user code; disk and memory are fine.

Anatomy of a command

$
commandvmstat

procps-ng vmstat: reads /proc/stat, /proc/meminfo and /proc/vmstat each interval.

How it works

vmstat reads kernel counters and prints the differences between samples. procs r and b come from /proc/stat (procs_running and procs_blocked: runnable tasks and tasks in uninterruptible sleep). Memory comes from /proc/meminfo. si/so (KiB/s swapped in from and out to disk) and bi/bo (blocks per second read from and written to block devices, 1 KiB units) come from /proc/vmstat counters. in and cs are interrupts and context switches per second; us, sy, id, wa, st and gu are percentages of CPU time, from the tick counters in /proc/stat. The first line is special: it is averaged since boot, so ignore it when diagnosing a current problem. vmstat costs almost nothing to run, which makes it safe on a box that is already struggling.

Options you'll actually use

DELAY [COUNT]Print every DELAY seconds, COUNT times (first line = since boot)
-S MMemory columns in MiB (k, K, m, M allowed)
-wWide output: wider columns that don't overflow
-tTimestamp on every line
-aShow active/inactive memory instead of buff/cache
-sOne-off table of memory and event counters
-dPer-disk read/write statistics
-ySkip the since-boot first line

Common mistakes

Reading the first line of vmstat 1 as the current state

The first line is an average since boot, often calm, while the problem is happening now.

Skip it, or use vmstat -y 1
Panicking at a non-zero swpd

swpd is how much is sitting in swap, possibly idle pages from days ago. That alone doesn't slow anything.

Watch si/so: sustained swap traffic is what hurts
Expecting bi/bo in megabytes

bi and bo are blocks per second, 1 KiB blocks on Linux. 180000 means about 180 MB/s.

Divide by 1024 for MiB/s, or use iostat -m
Comparing r with 1 instead of with the CPU count

r is the number of runnable tasks; a 16-CPU box can run 16 at once without queueing.

Sustained r above nproc means CPU saturation

Interview questions

Practice

Try these on a Linux machine of your own. Each task has a hint and a solution.

  1. 1

    Print 5 vmstat samples, one per second, with memory in MiB.

  2. 2

    Run vmstat for 3 samples without the since-boot line and with timestamps.

  3. 3

    Show the one-off summary of memory and event counters, and find how many forks happened since boot.

  4. 4

    Generate CPU load with `yes > /dev/null &` and watch r and us change in vmstat. Then stop it.

Related commands

iostatMeasure disk throughput, latency and utilisationfreeShow used, free and available memorytopWatch CPU, memory and processes livesarCollect and replay system activity historyuptimeHow long the system has run, and its load average
PreviousuptimeNextiostat