tar --listed-incrementalFull and incremental backups with GNU tar
GNU tar's --listed-incremental keeps a snapshot file describing what was backed up, so each run archives only files that changed since the last one, and records deletions so a restore reproduces the tree exactly. A full backup plus a chain of small incrementals, with nothing but tar.
tar --listed-incremental=SNAPSHOT.snar -czf ARCHIVE.tar.gz [-C DIR] PATH...You set up a weekly full plus daily incremental backup of /srv/app using only tar.
tarGNU tar; the incremental format is a GNU extension.
The snapshot (.snar) file stores, for every directory, its device and inode numbers, its modification time, and the names it contained, plus the time of the run. On the first run the file doesn't exist, so everything is new: tar prints 'Directory is new' and makes a level-0 (full) archive. On the next run tar compares each directory with the snapshot and archives only files whose mtime or ctime is newer than the previous run (ctime means a chmod or chown also triggers a re-backup). For every directory it also writes a 'dumpdir' entry listing all names present now, each flagged Y (in this archive), N (unchanged, in an earlier archive) or D (subdirectory). When you restore with --listed-incremental=/dev/null, tar extracts the full archive, then each incremental in order, and uses the dumpdirs to delete files that no longer existed at that point ('tar: Deleting ...'). Because the device number is recorded, restoring onto a new disk or an LVM snapshot can make tar think every directory is new; --no-check-device avoids that.
-g FILE / --listed-incremental=FILESnapshot file: create a full or incremental archive and update FILE--listed-incremental=/dev/nullOn extract/list: treat the archive as incremental (apply deletions)-G / --incrementalSame as above for extraction, in short form--level=0Force a full backup while still writing the snapshot file--no-check-deviceIgnore device numbers (new disk, LVM or NFS)-tvv with -g /dev/nullList archive with dumpdir contents (Y/N/D flags)-C DIRArchive relative to DIR; extract into DIR--exclude=PATTERNSkip caches and temp files--warning=no-file-changedSilence 'file changed as we read it' for live logsExtracting incrementals with plain tar -xzfFiles deleted between backups aren't removed, so the restored tree contains files that shouldn't exist.
tar --listed-incremental=/dev/null -xzf ... for every archive in the chainRestoring the incrementals before (or without) the full backupEach incremental only holds changes; without the level 0 most files are missing.
Full first, then each incremental oldest to newestLosing or overwriting the .snar fileThe next run has no state and either makes a full backup or compares against the wrong point.
Keep the .snar with the backups and copy the level-0 snapshot before each chainBacking up a live database directory this wayFiles change mid-read ('file changed as we read it') and the result is inconsistent.
pg_dump/mysqldump first, then back up the dump filesTry these on a Linux machine of your own. Each task has a hint and a solution.
Make a full incremental-format backup of ~/app into ~/bk/full.tar.gz with the snapshot file ~/bk/app.snar.
Create a new file in ~/app, delete another one, and make an incremental backup ~/bk/inc1.tar.gz.
Show the dumpdir information of inc1.tar.gz to see which files are Y and N.
Restore both archives into /tmp/rest and check that README.md was deleted.
tar --listed-incrementalFull and incremental backups with GNU tar
GNU tar's --listed-incremental keeps a snapshot file describing what was backed up, so each run archives only files that changed since the last one, and records deletions so a restore reproduces the tree exactly. A full backup plus a chain of small incrementals, with nothing but tar.
tar --listed-incremental=SNAPSHOT.snar -czf ARCHIVE.tar.gz [-C DIR] PATH...You set up a weekly full plus daily incremental backup of /srv/app using only tar.
tarGNU tar; the incremental format is a GNU extension.
The snapshot (.snar) file stores, for every directory, its device and inode numbers, its modification time, and the names it contained, plus the time of the run. On the first run the file doesn't exist, so everything is new: tar prints 'Directory is new' and makes a level-0 (full) archive. On the next run tar compares each directory with the snapshot and archives only files whose mtime or ctime is newer than the previous run (ctime means a chmod or chown also triggers a re-backup). For every directory it also writes a 'dumpdir' entry listing all names present now, each flagged Y (in this archive), N (unchanged, in an earlier archive) or D (subdirectory). When you restore with --listed-incremental=/dev/null, tar extracts the full archive, then each incremental in order, and uses the dumpdirs to delete files that no longer existed at that point ('tar: Deleting ...'). Because the device number is recorded, restoring onto a new disk or an LVM snapshot can make tar think every directory is new; --no-check-device avoids that.
-g FILE / --listed-incremental=FILESnapshot file: create a full or incremental archive and update FILE--listed-incremental=/dev/nullOn extract/list: treat the archive as incremental (apply deletions)-G / --incrementalSame as above for extraction, in short form--level=0Force a full backup while still writing the snapshot file--no-check-deviceIgnore device numbers (new disk, LVM or NFS)-tvv with -g /dev/nullList archive with dumpdir contents (Y/N/D flags)-C DIRArchive relative to DIR; extract into DIR--exclude=PATTERNSkip caches and temp files--warning=no-file-changedSilence 'file changed as we read it' for live logsExtracting incrementals with plain tar -xzfFiles deleted between backups aren't removed, so the restored tree contains files that shouldn't exist.
tar --listed-incremental=/dev/null -xzf ... for every archive in the chainRestoring the incrementals before (or without) the full backupEach incremental only holds changes; without the level 0 most files are missing.
Full first, then each incremental oldest to newestLosing or overwriting the .snar fileThe next run has no state and either makes a full backup or compares against the wrong point.
Keep the .snar with the backups and copy the level-0 snapshot before each chainBacking up a live database directory this wayFiles change mid-read ('file changed as we read it') and the result is inconsistent.
pg_dump/mysqldump first, then back up the dump filesTry these on a Linux machine of your own. Each task has a hint and a solution.
Make a full incremental-format backup of ~/app into ~/bk/full.tar.gz with the snapshot file ~/bk/app.snar.
Create a new file in ~/app, delete another one, and make an incremental backup ~/bk/inc1.tar.gz.
Show the dumpdir information of inc1.tar.gz to see which files are Y and N.
Restore both archives into /tmp/rest and check that README.md was deleted.