CrackOps
Interview PrepTracksLinuxAWS LabSoonInterview RoomSoon
Sign inStart free
CrackOps

Interview prep for the people who keep production up.

[email protected]
ProductInterview PrepTracksDashboardInterview Room
RolesDevOpsSREPlatformCloud
CompanyAboutContactPrivacyTerms
© CrackOps
Linux / Text Processing / sort

sort

Sort lines: alphabetically, numerically, by column

Beginnerpipelinesrankingessential

sort orders lines of text: alphabetically by default, or numerically (-n), by human sizes like 1.5G (-h), by version (-V), by a chosen column (-k) and in reverse (-r). It is the middle of almost every 'top N' pipeline and the required step before uniq.

Syntaxsort [OPTION]... [FILE]...

Watch it run

You want the biggest responses nginx sent, and a list of backend IPs in a sensible order.

devops@devops-lab: ~/logs — zsh — 96×28
➜ ~/logs
Step 1 of 3
awk '{print $10}' access.log | sort -u | tail -3

Wrong answer: plain sort compares text character by character, so '9998' comes after '184213' because 9 > 1. The 'largest' values are not the largest numbers.

Anatomy of a command

$
commandsort

Reads all lines, orders them, writes them out. Nothing is printed until the whole input has been read.

How it works

sort has to see every line before it can print the first one, so it reads the whole input into a memory buffer. When the data is bigger than that buffer (sized from available RAM, or set with -S), it sorts the chunk, writes it to a temporary file in $TMPDIR (default /tmp, change with -T), and at the end does a k-way merge of all the sorted runs: a classic external merge sort, using several threads on big inputs. Comparing two lines is where the options matter: by default it uses the locale's collation rules (strcoll), which for en_US.UTF-8 mostly ignores case and punctuation. LC_ALL=C makes it compare raw bytes, which is both predictable and much faster. -n parses a leading number, -h understands K/M/G suffixes, and -k restricts the comparison to certain fields. If keys tie, sort falls back to comparing the whole line, unless -s (stable) keeps the input order.

Options you'll actually use

-nNumeric sort: 9 before 10 (default text order puts 10 first)
-hHuman numeric: 980K < 120M < 1.5G (for du -h and ls -lh output)
-VVersion sort: 10.0.2.9 < 10.0.2.11, v1.9 < v1.10
-rReverse the order
-k N,MSort by fields N to M; add n, r, h... after it for that key only (-k2,2nr)
-t CField separator for -k (e.g. -t, for CSV, -t: for passwd)
-uOutput only one of each run of equal lines (or equal keys)
-o FILEWrite output to FILE; safe to use the input file
-cCheck whether input is sorted; report the first disorder
-sStable: keep the input order of lines with equal keys
-fFold case: treat a and A as equal
-S SIZE / -T DIRMemory buffer size / directory for temp files (huge inputs)

Common mistakes

sort -k2 file to sort by the second column

-k2 means 'from field 2 to the end of the line', so later fields take part in the comparison and break ties in surprising ways.

sort -k2,2 file # numeric: -k2,2n
sort users.txt > users.txt

The shell truncates users.txt before sort starts reading it, so you end up with an empty file.

sort -o users.txt users.txt
du -sh * | sort -n

-n reads only the digits and ignores K, M and G, so 980K sorts above 11G.

du -sh * | sort -h
Sorting with one locale and comparing with comm or join under another

Different collation orders make comm and join silently miss matches or complain the input isn't sorted.

export LC_ALL=C # before every sort | comm/join step

Interview questions

Practice

Try these on a Linux machine of your own. Each task has a hint and a solution.

  1. 1

    Sort ~/data/servers.txt by IP address (field 2) correctly, so .9 comes before .11.

  2. 2

    Show the 5 largest entries in ~/logs, human-readable, biggest first.

  3. 3

    Print the users in /etc/passwd sorted by UID, highest first, showing only name and UID.

  4. 4

    Sort ~/data/hosts.txt in place, removing duplicates.

Related commands

uniqCollapse and count repeated adjacent linesawkProcess text by columns: filter, sum, count and reportduFind out which files and directories use the spacecutCut columns or character ranges out of each line
PreviouscutNextuniq
Linux / Text Processing / sort

sort

Sort lines: alphabetically, numerically, by column

Beginnerpipelinesrankingessential

sort orders lines of text: alphabetically by default, or numerically (-n), by human sizes like 1.5G (-h), by version (-V), by a chosen column (-k) and in reverse (-r). It is the middle of almost every 'top N' pipeline and the required step before uniq.

Syntaxsort [OPTION]... [FILE]...

Watch it run

You want the biggest responses nginx sent, and a list of backend IPs in a sensible order.

devops@devops-lab: ~/logs — zsh — 96×28
➜ ~/logs
Step 1 of 3
awk '{print $10}' access.log | sort -u | tail -3

Wrong answer: plain sort compares text character by character, so '9998' comes after '184213' because 9 > 1. The 'largest' values are not the largest numbers.

Anatomy of a command

$
commandsort

Reads all lines, orders them, writes them out. Nothing is printed until the whole input has been read.

How it works

sort has to see every line before it can print the first one, so it reads the whole input into a memory buffer. When the data is bigger than that buffer (sized from available RAM, or set with -S), it sorts the chunk, writes it to a temporary file in $TMPDIR (default /tmp, change with -T), and at the end does a k-way merge of all the sorted runs: a classic external merge sort, using several threads on big inputs. Comparing two lines is where the options matter: by default it uses the locale's collation rules (strcoll), which for en_US.UTF-8 mostly ignores case and punctuation. LC_ALL=C makes it compare raw bytes, which is both predictable and much faster. -n parses a leading number, -h understands K/M/G suffixes, and -k restricts the comparison to certain fields. If keys tie, sort falls back to comparing the whole line, unless -s (stable) keeps the input order.

Options you'll actually use

-nNumeric sort: 9 before 10 (default text order puts 10 first)
-hHuman numeric: 980K < 120M < 1.5G (for du -h and ls -lh output)
-VVersion sort: 10.0.2.9 < 10.0.2.11, v1.9 < v1.10
-rReverse the order
-k N,MSort by fields N to M; add n, r, h... after it for that key only (-k2,2nr)
-t CField separator for -k (e.g. -t, for CSV, -t: for passwd)
-uOutput only one of each run of equal lines (or equal keys)
-o FILEWrite output to FILE; safe to use the input file
-cCheck whether input is sorted; report the first disorder
-sStable: keep the input order of lines with equal keys
-fFold case: treat a and A as equal
-S SIZE / -T DIRMemory buffer size / directory for temp files (huge inputs)

Common mistakes

sort -k2 file to sort by the second column

-k2 means 'from field 2 to the end of the line', so later fields take part in the comparison and break ties in surprising ways.

sort -k2,2 file # numeric: -k2,2n
sort users.txt > users.txt

The shell truncates users.txt before sort starts reading it, so you end up with an empty file.

sort -o users.txt users.txt
du -sh * | sort -n

-n reads only the digits and ignores K, M and G, so 980K sorts above 11G.

du -sh * | sort -h
Sorting with one locale and comparing with comm or join under another

Different collation orders make comm and join silently miss matches or complain the input isn't sorted.

export LC_ALL=C # before every sort | comm/join step

Interview questions

Practice

Try these on a Linux machine of your own. Each task has a hint and a solution.

  1. 1

    Sort ~/data/servers.txt by IP address (field 2) correctly, so .9 comes before .11.

  2. 2

    Show the 5 largest entries in ~/logs, human-readable, biggest first.

  3. 3

    Print the users in /etc/passwd sorted by UID, highest first, showing only name and UID.

  4. 4

    Sort ~/data/hosts.txt in place, removing duplicates.

Related commands

uniqCollapse and count repeated adjacent linesawkProcess text by columns: filter, sum, count and reportduFind out which files and directories use the spacecutCut columns or character ranges out of each line
PreviouscutNextuniq