rsyncSync files and directories efficiently
rsync makes a destination match a source, locally or over SSH, sending only what changed and only the changed parts of changed files. With -a it keeps permissions, times and links; with --delete it mirrors. It's the workhorse for deployments, backups and moving large trees.
rsync [-avzh] [-n] [--delete] [--exclude=PAT] [--progress] SRC... [USER@]HOST:DEST (or local DEST)Back up an app directory to a second disk, and see why the second run is almost instant.
rsyncRemote sync. Runs a second rsync on the other side (over ssh) and the two compare notes.
For a remote copy rsync runs ssh host rsync --server ... and the two processes talk over that pipe. The sender walks the source and sends a file list (names, sizes, modification times, modes). On the destination the generator compares each entry with what's there; by default a file whose size and mtime both match is skipped without reading it (the 'quick check'; -c compares checksums instead). For a file that differs, the generator splits the existing destination copy into blocks and sends a weak rolling checksum and a strong hash for each. The sender slides over its version byte by byte, finds blocks the destination already has, and sends only references to those plus the literal bytes that are new. The receiver rebuilds the file into a hidden temporary file next to the target and renames it into place, so readers never see half a file. New files are sent whole, -z compresses the stream, and --delete removes destination entries missing from the source.
The receiver describes what it already has with block checksums, and the sender transmits only the bytes that are new.
rsync runs ssh app-1 rsync --server. The two rsync processes talk over that encrypted pipe; on the remote side it splits into a generator and a receiver.
-aArchive: -rlptgoD (recursive, links, perms, times, group, owner, devices)-v / -hList transferred files / human-readable numbers-zCompress data in transit (good over slow links, not on a LAN)-n / --dry-runShow what would happen, change nothing--deleteDelete destination files that aren't in the source--exclude=PATSkip paths matching PAT (repeatable; --exclude-from=FILE)-iItemize: one code per file saying what changed and why--progress / -PPer-file progress / --partial plus --progress (resume big files)--info=progress2One overall progress line for the whole transfer--statsSummary: files, literal vs matched data, bytes sent-e 'ssh -p 2222'Remote shell command (custom port, key, options)--rsync-path='sudo rsync'Run the remote rsync with sudo (write root-owned paths)-cCompare by checksum instead of size and mtime (slower, thorough)--bwlimit=RATELimit bandwidth (e.g. 20m)rsync -av --delete build /var/www/site/Without the trailing slash the source directory itself is copied, creating /var/www/site/build, and --delete then wipes everything else in /var/www/site.
rsync -av --delete build/ /var/www/site/ # and dry-run with -n firstRunning --delete for the first time without -nA wrong path or slash deletes data at the destination with no undo.
rsync -ain --delete src/ dest/ # read every *deleting line, then rerun without -nUsing -z on a fast LAN or for already-compressed filesCompression burns CPU on both ends and can make the transfer slower; .gz, .jpg and .mp4 don't shrink.
Use -z over slow WAN links only.Assuming rsync caught a change in a file edited within the same secondThe quick check compares size and mtime; a same-size change with an identical timestamp (copied or restored files) is skipped.
rsync -ac ... when correctness matters more than speedIgnoring exit code 23 or 24 in backup scripts23 means some files weren't transferred (permissions, I/O); 24 means files vanished during the run. The backup is incomplete.
Check $? and alert on anything but 0 (and decide whether 24 is acceptable).Try these on a Linux machine of your own. Each task has a hint and a solution.
Copy the contents of ~/app into /tmp/app-backup with archive mode, listing the files.
Run the same command again. What changed in the output and in the speedup figure?
Create an extra file /tmp/app-backup/stale.txt, then show (without changing anything) what a mirror sync would delete.
Copy ~/logs to /tmp/logs-nolog, excluding every *.gz file, with human-readable stats.
rsyncSync files and directories efficiently
rsync makes a destination match a source, locally or over SSH, sending only what changed and only the changed parts of changed files. With -a it keeps permissions, times and links; with --delete it mirrors. It's the workhorse for deployments, backups and moving large trees.
rsync [-avzh] [-n] [--delete] [--exclude=PAT] [--progress] SRC... [USER@]HOST:DEST (or local DEST)Back up an app directory to a second disk, and see why the second run is almost instant.
rsyncRemote sync. Runs a second rsync on the other side (over ssh) and the two compare notes.
For a remote copy rsync runs ssh host rsync --server ... and the two processes talk over that pipe. The sender walks the source and sends a file list (names, sizes, modification times, modes). On the destination the generator compares each entry with what's there; by default a file whose size and mtime both match is skipped without reading it (the 'quick check'; -c compares checksums instead). For a file that differs, the generator splits the existing destination copy into blocks and sends a weak rolling checksum and a strong hash for each. The sender slides over its version byte by byte, finds blocks the destination already has, and sends only references to those plus the literal bytes that are new. The receiver rebuilds the file into a hidden temporary file next to the target and renames it into place, so readers never see half a file. New files are sent whole, -z compresses the stream, and --delete removes destination entries missing from the source.
The receiver describes what it already has with block checksums, and the sender transmits only the bytes that are new.
rsync runs ssh app-1 rsync --server. The two rsync processes talk over that encrypted pipe; on the remote side it splits into a generator and a receiver.
-aArchive: -rlptgoD (recursive, links, perms, times, group, owner, devices)-v / -hList transferred files / human-readable numbers-zCompress data in transit (good over slow links, not on a LAN)-n / --dry-runShow what would happen, change nothing--deleteDelete destination files that aren't in the source--exclude=PATSkip paths matching PAT (repeatable; --exclude-from=FILE)-iItemize: one code per file saying what changed and why--progress / -PPer-file progress / --partial plus --progress (resume big files)--info=progress2One overall progress line for the whole transfer--statsSummary: files, literal vs matched data, bytes sent-e 'ssh -p 2222'Remote shell command (custom port, key, options)--rsync-path='sudo rsync'Run the remote rsync with sudo (write root-owned paths)-cCompare by checksum instead of size and mtime (slower, thorough)--bwlimit=RATELimit bandwidth (e.g. 20m)rsync -av --delete build /var/www/site/Without the trailing slash the source directory itself is copied, creating /var/www/site/build, and --delete then wipes everything else in /var/www/site.
rsync -av --delete build/ /var/www/site/ # and dry-run with -n firstRunning --delete for the first time without -nA wrong path or slash deletes data at the destination with no undo.
rsync -ain --delete src/ dest/ # read every *deleting line, then rerun without -nUsing -z on a fast LAN or for already-compressed filesCompression burns CPU on both ends and can make the transfer slower; .gz, .jpg and .mp4 don't shrink.
Use -z over slow WAN links only.Assuming rsync caught a change in a file edited within the same secondThe quick check compares size and mtime; a same-size change with an identical timestamp (copied or restored files) is skipped.
rsync -ac ... when correctness matters more than speedIgnoring exit code 23 or 24 in backup scripts23 means some files weren't transferred (permissions, I/O); 24 means files vanished during the run. The backup is incomplete.
Check $? and alert on anything but 0 (and decide whether 24 is acceptable).Try these on a Linux machine of your own. Each task has a hint and a solution.
Copy the contents of ~/app into /tmp/app-backup with archive mode, listing the files.
Run the same command again. What changed in the output and in the speedup figure?
Create an extra file /tmp/app-backup/stale.txt, then show (without changing anything) what a mirror sync would delete.
Copy ~/logs to /tmp/logs-nolog, excluding every *.gz file, with human-readable stats.