iostatMeasure disk throughput, latency and utilisation
iostat (from the sysstat package) reports CPU usage and per-device disk statistics: requests per second, throughput, average latency (await), queue size and %util. With -x it is the standard tool for answering 'is the disk the bottleneck, and which one?'.
iostat [-c] [-d] [-x] [-s] [-z] [-y] [-h|-k|-m] [-p DEV] [DEVICE...] [INTERVAL [COUNT]]You're investigating slow queries and want disk statistics, but iostat isn't on this minimal image.
iostatsysstat's iostat: reads /proc/diskstats and /proc/stat and prints rates per interval.
The kernel keeps per-device counters in /proc/diskstats: reads and writes completed, sectors transferred, merged requests, milliseconds spent on reads and writes, requests in flight, and milliseconds the device had at least one request in flight. iostat samples them every interval and divides the differences by the elapsed time. r/s and w/s are completed requests per second; rkB/s and wkB/s throughput; await (r_await, w_await) is the average time from submission to completion, including time queued in the kernel, in ms; aqu-sz is the average number of requests in flight; %util is the share of time the device was busy. %util is accurate for a single-queue disk, but SSDs and NVMe serve many requests in parallel, so they can show 100% while having headroom: latency (await) and queue size tell you more. The avg-cpu block, including %iowait, comes from /proc/stat.
-xExtended stats: r/s, w/s, rkB/s, wkB/s, r_await, w_await, aqu-sz, %util-sShort version of -x that fits 80 columns: tps, kB/s, await, %util-d / -cOnly the device report / only the CPU report-zOmit devices with no activity-ySkip the since-boot first report-h / -mHuman-readable units / MB instead of kB-p DEVInclude partitions of DEV-tPrint a timestamp with each report-NShow LVM/device-mapper names instead of dm-0Running iostat with no interval during an incidentWithout an interval you get one report averaged since boot, which smooths away the current problem.
iostat -dxz 1 5, and read from the second report on (or add -y)Concluding an NVMe disk is maxed out because %util is 100SSDs and NVMe process many requests in parallel; %util only says at least one request was in flight all the time.
Judge by await and aqu-sz against the device's normal latencyBlaming the disk for high %iowait aloneiowait is idle CPU time while some I/O is outstanding; it shrinks when other work keeps the CPUs busy, and varies with CPU count.
Check per-device await, aqu-sz and throughput with iostat -xTry these on a Linux machine of your own. Each task has a hint and a solution.
Show extended device statistics every second, 3 times, hiding idle devices.
Show just the CPU report, skipping the since-boot sample, 2 times at 2-second intervals.
Generate disk writes with dd and watch the device in iostat at the same time.
iostatMeasure disk throughput, latency and utilisation
iostat (from the sysstat package) reports CPU usage and per-device disk statistics: requests per second, throughput, average latency (await), queue size and %util. With -x it is the standard tool for answering 'is the disk the bottleneck, and which one?'.
iostat [-c] [-d] [-x] [-s] [-z] [-y] [-h|-k|-m] [-p DEV] [DEVICE...] [INTERVAL [COUNT]]You're investigating slow queries and want disk statistics, but iostat isn't on this minimal image.
iostatsysstat's iostat: reads /proc/diskstats and /proc/stat and prints rates per interval.
The kernel keeps per-device counters in /proc/diskstats: reads and writes completed, sectors transferred, merged requests, milliseconds spent on reads and writes, requests in flight, and milliseconds the device had at least one request in flight. iostat samples them every interval and divides the differences by the elapsed time. r/s and w/s are completed requests per second; rkB/s and wkB/s throughput; await (r_await, w_await) is the average time from submission to completion, including time queued in the kernel, in ms; aqu-sz is the average number of requests in flight; %util is the share of time the device was busy. %util is accurate for a single-queue disk, but SSDs and NVMe serve many requests in parallel, so they can show 100% while having headroom: latency (await) and queue size tell you more. The avg-cpu block, including %iowait, comes from /proc/stat.
-xExtended stats: r/s, w/s, rkB/s, wkB/s, r_await, w_await, aqu-sz, %util-sShort version of -x that fits 80 columns: tps, kB/s, await, %util-d / -cOnly the device report / only the CPU report-zOmit devices with no activity-ySkip the since-boot first report-h / -mHuman-readable units / MB instead of kB-p DEVInclude partitions of DEV-tPrint a timestamp with each report-NShow LVM/device-mapper names instead of dm-0Running iostat with no interval during an incidentWithout an interval you get one report averaged since boot, which smooths away the current problem.
iostat -dxz 1 5, and read from the second report on (or add -y)Concluding an NVMe disk is maxed out because %util is 100SSDs and NVMe process many requests in parallel; %util only says at least one request was in flight all the time.
Judge by await and aqu-sz against the device's normal latencyBlaming the disk for high %iowait aloneiowait is idle CPU time while some I/O is outstanding; it shrinks when other work keeps the CPUs busy, and varies with CPU count.
Check per-device await, aqu-sz and throughput with iostat -xTry these on a Linux machine of your own. Each task has a hint and a solution.
Show extended device statistics every second, 3 times, hiding idle devices.
Show just the CPU report, skipping the since-boot sample, 2 times at 2-second intervals.
Generate disk writes with dd and watch the device in iostat at the same time.