xargsBuild and run commands from standard input
xargs reads items from stdin and runs a command with those items as arguments, packing as many as fit into each run. It's the bridge between commands that print lists (find, grep -l, cut) and commands that take arguments (rm, gzip, ssh), with options for safe file names (-0), one item per run (-n1, -I{}) and parallelism (-P).
xargs [-0] [-r] [-n MAX] [-P PROCS] [-I REPLSTR] [COMMAND [INITIAL-ARGS]...]See how xargs turns a list into command-line arguments before you point it at anything destructive.
find /var/log/app -name '*.log.1'The producer: prints the paths of rotated logs, one item per file.
xargs reads stdin and splits it into items: by default on blanks and newlines, honouring quotes and backslashes (which is why a name like O'Brien breaks it), or on NUL bytes with -0, or on lines with -d '\n'. It appends items to the command's argument list until it reaches a limit: -n items, or the size of its command buffer (128 KiB by default on Linux, bounded by the kernel's ARG_MAX of about 2 MB). Then it fork()s, execve()s the command with that argv and waits for it, unless -P lets it keep several children running. It repeats until input is exhausted. Packing many arguments into one execve is why find | xargs rm is far faster than running rm once per file, and why it avoids the shell's 'Argument list too long'. The exit status summarises the children: 0 if all succeeded, 123 if any returned 1-125, 124 if one returned 255, 125 if killed by a signal, 127 if the command wasn't found.
xargs packs input items into argument lists, then forks and execs the command once per batch, several at a time with -P.
find writes each matching path followed by a NUL byte into the pipe. It doesn't wait for xargs; both run at the same time.
-0Items are NUL-separated (pair with find -print0, grep -lZ, sort -z)-d '\n'Split on newlines only; quotes and spaces are literal (GNU)-rDon't run the command if stdin is empty (GNU; BSD does this by default)-n NUse at most N items per command run-I {}Run once per input line, replacing {} anywhere in the command-P NRun up to N commands in parallel (0 = as many as possible)-tPrint each command to stderr before running it (trace)-pPrompt for confirmation before each run-L NUse at most N input lines per command run--show-limitsPrint the argument size limits xargs works withfind . -name '*.log' | xargs rmNames with spaces are split into several arguments and quotes break parsing, so the wrong paths are removed or nothing is.
find . -name '*.log' -print0 | xargs -0 rm # or: find . -name '*.log' -deletels *.gz | xargs -I{} rm {} for 100,000 files-I runs one process per line, so 100,000 forks. It's slow, and parsing ls output breaks on unusual names.
find . -name '*.gz' -print0 | xargs -0 rmgrep -l secret *.yml | xargs sed -i 's/secret/REDACTED/' in a cron jobWhen nothing matches, GNU xargs still runs sed -i with no file and it fails with 'no input files', making the job error every night.
grep -lZ secret *.yml | xargs -0 -r sed -i 's/secret/REDACTED/'cat hosts | xargs -P20 -I{} ssh {} uptime and reading the resultsParallel outputs interleave on the terminal and you can't tell which host said what.
xargs -P20 -I{} sh -c 'ssh {} uptime > out/{}.txt 2>&1' < hostsTry these on a Linux machine of your own. Each task has a hint and a solution.
Count the lines of every .py file under ~/app with a single wc command.
Create the files 'a b.txt' and c.txt in ~/tmp, then delete all .txt files there safely with find and xargs.
Print a dry-run 'ping -c1 HOST' command for every host in ~/data/hosts.txt (the sandbox has no network).
Compress every .log file in /var/log/app into .gz, running 4 gzip processes at a time, and don't run anything if there are none.
xargsBuild and run commands from standard input
xargs reads items from stdin and runs a command with those items as arguments, packing as many as fit into each run. It's the bridge between commands that print lists (find, grep -l, cut) and commands that take arguments (rm, gzip, ssh), with options for safe file names (-0), one item per run (-n1, -I{}) and parallelism (-P).
xargs [-0] [-r] [-n MAX] [-P PROCS] [-I REPLSTR] [COMMAND [INITIAL-ARGS]...]See how xargs turns a list into command-line arguments before you point it at anything destructive.
find /var/log/app -name '*.log.1'The producer: prints the paths of rotated logs, one item per file.
xargs reads stdin and splits it into items: by default on blanks and newlines, honouring quotes and backslashes (which is why a name like O'Brien breaks it), or on NUL bytes with -0, or on lines with -d '\n'. It appends items to the command's argument list until it reaches a limit: -n items, or the size of its command buffer (128 KiB by default on Linux, bounded by the kernel's ARG_MAX of about 2 MB). Then it fork()s, execve()s the command with that argv and waits for it, unless -P lets it keep several children running. It repeats until input is exhausted. Packing many arguments into one execve is why find | xargs rm is far faster than running rm once per file, and why it avoids the shell's 'Argument list too long'. The exit status summarises the children: 0 if all succeeded, 123 if any returned 1-125, 124 if one returned 255, 125 if killed by a signal, 127 if the command wasn't found.
xargs packs input items into argument lists, then forks and execs the command once per batch, several at a time with -P.
find writes each matching path followed by a NUL byte into the pipe. It doesn't wait for xargs; both run at the same time.
-0Items are NUL-separated (pair with find -print0, grep -lZ, sort -z)-d '\n'Split on newlines only; quotes and spaces are literal (GNU)-rDon't run the command if stdin is empty (GNU; BSD does this by default)-n NUse at most N items per command run-I {}Run once per input line, replacing {} anywhere in the command-P NRun up to N commands in parallel (0 = as many as possible)-tPrint each command to stderr before running it (trace)-pPrompt for confirmation before each run-L NUse at most N input lines per command run--show-limitsPrint the argument size limits xargs works withfind . -name '*.log' | xargs rmNames with spaces are split into several arguments and quotes break parsing, so the wrong paths are removed or nothing is.
find . -name '*.log' -print0 | xargs -0 rm # or: find . -name '*.log' -deletels *.gz | xargs -I{} rm {} for 100,000 files-I runs one process per line, so 100,000 forks. It's slow, and parsing ls output breaks on unusual names.
find . -name '*.gz' -print0 | xargs -0 rmgrep -l secret *.yml | xargs sed -i 's/secret/REDACTED/' in a cron jobWhen nothing matches, GNU xargs still runs sed -i with no file and it fails with 'no input files', making the job error every night.
grep -lZ secret *.yml | xargs -0 -r sed -i 's/secret/REDACTED/'cat hosts | xargs -P20 -I{} ssh {} uptime and reading the resultsParallel outputs interleave on the terminal and you can't tell which host said what.
xargs -P20 -I{} sh -c 'ssh {} uptime > out/{}.txt 2>&1' < hostsTry these on a Linux machine of your own. Each task has a hint and a solution.
Count the lines of every .py file under ~/app with a single wc command.
Create the files 'a b.txt' and c.txt in ~/tmp, then delete all .txt files there safely with find and xargs.
Print a dry-run 'ping -c1 HOST' command for every host in ~/data/hosts.txt (the sandbox has no network).
Compress every .log file in /var/log/app into .gz, running 4 gzip processes at a time, and don't run anything if there are none.