gzipCompress and decompress single files with gzip
gzip compresses a file in place, replacing access.log with access.log.gz, and gunzip reverses it. It's fast, everywhere, and the format behind .tar.gz, rotated logs, HTTP compression and database dumps; zcat and zgrep read the compressed files without unpacking them.
gzip [OPTION]... [FILE]... | gunzip FILE.gz | zcat FILE.gzOld access logs are taking space; compress yesterday's 224M log and see how much you saved.
gzipGNU zip: DEFLATE compression of one file into FILE.gz.
gzip uses DEFLATE: LZ77 replaces repeated byte sequences with back-references into the previous 32 KiB, then Huffman coding gives frequent symbols shorter bit codes. Text logs, with their repeated IPs, paths and user agents, typically shrink by 85-95%; JPEGs and already-compressed data don't shrink at all. The output file has a 10-byte header (magic 1f 8b, method, original mtime and usually the name), the compressed blocks, and a trailer with a CRC32 and the original length; gzip -t and gunzip check both. gzip writes FILE.gz, copies the original's mode and timestamps to it, then unlinks the original unless you use -k or -c. Compression is single-threaded; pigz splits the input into chunks and compresses them on all cores into a normal gzip file. Concatenated .gz files are still a valid gzip file, which is why zcat works on appended logs.
-kKeep the original file-dDecompress (same as gunzip)-cWrite to stdout, leave files alone: gzip -c f > f.gz-1 ... -9Speed vs size; --fast = -1, --best = -9, default -6-lList compressed size, original size and ratio-tTest integrity (CRC and length) without writing anything-rRecurse into directories and compress every file separately-vVerbose: show the reduction per file-fForce: overwrite existing .gz, compress links-nDon't store the name and timestamp (reproducible output)zcat / zgrep / zlessRead, search and page .gz files without extractinggzip app/ expecting an archive of the directorygzip only compresses files: it says 'is a directory -- ignored', and gzip -r compresses every file inside separately.
tar -czf app.tar.gz appgzip on a log file the application still writes togzip deletes the original, but the app keeps writing to the deleted inode, so new lines are lost and space isn't freed.
Let logrotate rotate first (create or copytruncate), then compress the rotated filegzip -9 everywhere because it's 'best'It costs several times the CPU for a few percent; on big nightly jobs that's minutes of runtime.
Default level, pigz for speed, or zstd for a better ratio at speedgunzip -c big.sql.gz > big.sql just to grep itDoubles the disk use for nothing.
zgrep 'pattern' big.sql.gz or zcat big.sql.gz | grep ...Try these on a Linux machine of your own. Each task has a hint and a solution.
Compress a copy of one log in ~/logs while keeping the original, then compare the sizes.
Show the compression ratio of the .gz file you made.
Count the lines containing ERROR in the compressed file without decompressing it to disk.
Test the integrity of every .gz file in ~/logs.
gzipCompress and decompress single files with gzip
gzip compresses a file in place, replacing access.log with access.log.gz, and gunzip reverses it. It's fast, everywhere, and the format behind .tar.gz, rotated logs, HTTP compression and database dumps; zcat and zgrep read the compressed files without unpacking them.
gzip [OPTION]... [FILE]... | gunzip FILE.gz | zcat FILE.gzOld access logs are taking space; compress yesterday's 224M log and see how much you saved.
gzipGNU zip: DEFLATE compression of one file into FILE.gz.
gzip uses DEFLATE: LZ77 replaces repeated byte sequences with back-references into the previous 32 KiB, then Huffman coding gives frequent symbols shorter bit codes. Text logs, with their repeated IPs, paths and user agents, typically shrink by 85-95%; JPEGs and already-compressed data don't shrink at all. The output file has a 10-byte header (magic 1f 8b, method, original mtime and usually the name), the compressed blocks, and a trailer with a CRC32 and the original length; gzip -t and gunzip check both. gzip writes FILE.gz, copies the original's mode and timestamps to it, then unlinks the original unless you use -k or -c. Compression is single-threaded; pigz splits the input into chunks and compresses them on all cores into a normal gzip file. Concatenated .gz files are still a valid gzip file, which is why zcat works on appended logs.
-kKeep the original file-dDecompress (same as gunzip)-cWrite to stdout, leave files alone: gzip -c f > f.gz-1 ... -9Speed vs size; --fast = -1, --best = -9, default -6-lList compressed size, original size and ratio-tTest integrity (CRC and length) without writing anything-rRecurse into directories and compress every file separately-vVerbose: show the reduction per file-fForce: overwrite existing .gz, compress links-nDon't store the name and timestamp (reproducible output)zcat / zgrep / zlessRead, search and page .gz files without extractinggzip app/ expecting an archive of the directorygzip only compresses files: it says 'is a directory -- ignored', and gzip -r compresses every file inside separately.
tar -czf app.tar.gz appgzip on a log file the application still writes togzip deletes the original, but the app keeps writing to the deleted inode, so new lines are lost and space isn't freed.
Let logrotate rotate first (create or copytruncate), then compress the rotated filegzip -9 everywhere because it's 'best'It costs several times the CPU for a few percent; on big nightly jobs that's minutes of runtime.
Default level, pigz for speed, or zstd for a better ratio at speedgunzip -c big.sql.gz > big.sql just to grep itDoubles the disk use for nothing.
zgrep 'pattern' big.sql.gz or zcat big.sql.gz | grep ...Try these on a Linux machine of your own. Each task has a hint and a solution.
Compress a copy of one log in ~/logs while keeping the original, then compare the sizes.
Show the compression ratio of the .gz file you made.
Count the lines containing ERROR in the compressed file without decompressing it to disk.
Test the integrity of every .gz file in ~/logs.