CrackOps
Interview PrepTracksLinuxAWS LabSoonInterview RoomSoon
Sign inStart free
CrackOps

Interview prep for the people who keep production up.

[email protected]
ProductInterview PrepTracksDashboardInterview Room
RolesDevOpsSREPlatformCloud
CompanyAboutContactPrivacyTerms
© CrackOps
Linux / Debugging & Diagnostics / lsof

lsof

List open files, sockets and the processes holding them

Intermediatedebuggingfilessockets

lsof lists open files and the processes that have them open. On Linux almost everything is a file, so it answers: who is using port 8080, which process holds this deleted log, why can't I unmount this disk, and how many descriptors has my app leaked.

Syntaxlsof [-nP] [-i [proto][@host][:port]] [-p PID] [-u USER] [-c NAME] [+L1] [-a] [FILE|DIR]...

Watch it run

Your service won't start because the port is taken, and you need to know which process holds it.

devops@devops-lab: ~/app — zsh — 96×28
➜ ~/app
Step 1 of 3
node server.js

Only one socket can listen on a port (unless they all use SO_REUSEPORT). Something is already bound to 8080.

Anatomy of a command

$
commandsudo

Without root lsof only sees your own processes' descriptors; other users' /proc/PID/fd is private.

How it works

lsof doesn't use a special kernel API: it walks /proc. For every process it reads the symlinks in /proc/PID/fd (each open descriptor), /proc/PID/fdinfo (offset and flags), /proc/PID/maps (memory-mapped libraries, shown as mem), and cwd, root and exe (shown as cwd, rtd, txt). It stats each target to fill TYPE, DEVICE, SIZE/OFF and NODE. A socket appears as socket:[inode]; lsof looks that inode up in /proc/net/tcp, tcp6, udp and unix to print addresses and the TCP state. A file that was deleted while still open has a link count of 0 but its inode and blocks live on until the last descriptor closes, which is why lsof shows (deleted) and why df and du can disagree. Reverse DNS for every address makes lsof slow: always use -n and -P on servers. With several selectors lsof ORs them; -a makes it AND.

Options you'll actually use

-i :PORTNetwork files on this port; -i TCP, -i @10.0.2.15, -i TCP:443
-sTCP:STATEWith -i: only this TCP state (LISTEN, ESTABLISHED, CLOSE_WAIT...)
-nPNo hostname or port-name lookups: faster and unambiguous
-p PIDEverything one process has open (comma-separate several PIDs)
-u USERFiles opened by this user; -u ^root excludes root
-c NAMEProcesses whose command name starts with NAME
-aAND the selectors (default is OR): -u app -a -i
+L1Open files with link count below 1: deleted but still held open
+D DIREverything open under DIR, recursively (slow on big trees)
-tPrint PIDs only, for scripts: kill $(lsof -t -i :8080)
-r SECRepeat every SEC seconds, like watch

Common mistakes

lsof -i :443 returns nothing, so 'nothing is listening'

Without root, lsof only shows your own processes. nginx runs as root/www-data, so you see an empty result.

sudo lsof -nP -i :443 (or sudo ss -ltnp 'sport = :443')
rm /var/log/app/app.log to free space while the app is running

The app keeps writing to the open, now nameless file. The space is not freed until the process closes it or exits.

truncate -s 0 /var/log/app/app.log (or : > file), then fix rotation
lsof -u app -i :8080 expecting 'app's sockets on 8080'

Selectors are ORed by default: you get every file of user app plus every socket on 8080.

lsof -a -u app -i :8080
Running plain lsof on a busy server

It lists every file of every process and does a DNS lookup per socket: slow, huge, and it can stall on a dead NFS mount.

Narrow it: lsof -nP -p PID, -i :PORT, or a path; use -b to avoid blocking kernel calls

Interview questions

Practice

Try these on a Linux machine of your own. Each task has a hint and a solution.

  1. 1

    List all the files your current shell has open.

  2. 2

    Start 'sleep 600 > ~/logs/held.log &', delete ~/logs/held.log, then prove the file still exists.

  3. 3

    Start 'python3 -m http.server 8000 &' and find its PID using only lsof, then stop it.

  4. 4

    Show only the TCP sockets that are in LISTEN state on this machine, with numeric addresses and ports.

Related commands

ssSee listening ports and open connectionsdfReport used and free space per filesystemduFind out which files and directories use the spaceulimitShow and set per-process resource limitsstraceTrace the system calls a process makes
PreviousltraceNextdmesg
Linux / Debugging & Diagnostics / lsof

lsof

List open files, sockets and the processes holding them

Intermediatedebuggingfilessockets

lsof lists open files and the processes that have them open. On Linux almost everything is a file, so it answers: who is using port 8080, which process holds this deleted log, why can't I unmount this disk, and how many descriptors has my app leaked.

Syntaxlsof [-nP] [-i [proto][@host][:port]] [-p PID] [-u USER] [-c NAME] [+L1] [-a] [FILE|DIR]...

Watch it run

Your service won't start because the port is taken, and you need to know which process holds it.

devops@devops-lab: ~/app — zsh — 96×28
➜ ~/app
Step 1 of 3
node server.js

Only one socket can listen on a port (unless they all use SO_REUSEPORT). Something is already bound to 8080.

Anatomy of a command

$
commandsudo

Without root lsof only sees your own processes' descriptors; other users' /proc/PID/fd is private.

How it works

lsof doesn't use a special kernel API: it walks /proc. For every process it reads the symlinks in /proc/PID/fd (each open descriptor), /proc/PID/fdinfo (offset and flags), /proc/PID/maps (memory-mapped libraries, shown as mem), and cwd, root and exe (shown as cwd, rtd, txt). It stats each target to fill TYPE, DEVICE, SIZE/OFF and NODE. A socket appears as socket:[inode]; lsof looks that inode up in /proc/net/tcp, tcp6, udp and unix to print addresses and the TCP state. A file that was deleted while still open has a link count of 0 but its inode and blocks live on until the last descriptor closes, which is why lsof shows (deleted) and why df and du can disagree. Reverse DNS for every address makes lsof slow: always use -n and -P on servers. With several selectors lsof ORs them; -a makes it AND.

Options you'll actually use

-i :PORTNetwork files on this port; -i TCP, -i @10.0.2.15, -i TCP:443
-sTCP:STATEWith -i: only this TCP state (LISTEN, ESTABLISHED, CLOSE_WAIT...)
-nPNo hostname or port-name lookups: faster and unambiguous
-p PIDEverything one process has open (comma-separate several PIDs)
-u USERFiles opened by this user; -u ^root excludes root
-c NAMEProcesses whose command name starts with NAME
-aAND the selectors (default is OR): -u app -a -i
+L1Open files with link count below 1: deleted but still held open
+D DIREverything open under DIR, recursively (slow on big trees)
-tPrint PIDs only, for scripts: kill $(lsof -t -i :8080)
-r SECRepeat every SEC seconds, like watch

Common mistakes

lsof -i :443 returns nothing, so 'nothing is listening'

Without root, lsof only shows your own processes. nginx runs as root/www-data, so you see an empty result.

sudo lsof -nP -i :443 (or sudo ss -ltnp 'sport = :443')
rm /var/log/app/app.log to free space while the app is running

The app keeps writing to the open, now nameless file. The space is not freed until the process closes it or exits.

truncate -s 0 /var/log/app/app.log (or : > file), then fix rotation
lsof -u app -i :8080 expecting 'app's sockets on 8080'

Selectors are ORed by default: you get every file of user app plus every socket on 8080.

lsof -a -u app -i :8080
Running plain lsof on a busy server

It lists every file of every process and does a DNS lookup per socket: slow, huge, and it can stall on a dead NFS mount.

Narrow it: lsof -nP -p PID, -i :PORT, or a path; use -b to avoid blocking kernel calls

Interview questions

Practice

Try these on a Linux machine of your own. Each task has a hint and a solution.

  1. 1

    List all the files your current shell has open.

  2. 2

    Start 'sleep 600 > ~/logs/held.log &', delete ~/logs/held.log, then prove the file still exists.

  3. 3

    Start 'python3 -m http.server 8000 &' and find its PID using only lsof, then stop it.

  4. 4

    Show only the TCP sockets that are in LISTEN state on this machine, with numeric addresses and ports.

Related commands

ssSee listening ports and open connectionsdfReport used and free space per filesystemduFind out which files and directories use the spaceulimitShow and set per-process resource limitsstraceTrace the system calls a process makes
PreviousltraceNextdmesg