Jump to content

Archiving and compression

From ArchWiki

(Redirected from Tar)

The traditional Unix archiving and compression tools are separated according to the Unix philosophy:

A file archiver combines several files into one archive file, e.g. tar.
A compression tool compresses and decompresses data, e.g. gzip.

These tools are often used in sequence by firstly creating an archive file and then compressing it.

Of course there are also tools that do both, which tend to additionally offer encryption, error detection and recovery.

Archiving only

Name	Package	Manuals	Description
GNU tar	tar	tar(1), info	Core utility for manipulating the ubiquitous tar archives (tarballs), which are used by pacman and the AUR.
libarchive	libarchive	bsdtar(1) bsdcpio(1)	Implementation of tar and cpio that also offers a library. Used by pacman and mkinitcpio.
ar	binutils	ar(1)	Legacy Unix archiver before tar. Today only used for creating static library files.
cpio	cpio	cpio(1)	File archiver via stdin/stdout, supports cpio and tar formats.

Tip: Both GNU and BSD tar automatically do decompression delegation for bzip2, compress, gzip, lzip, lzma, lzop, zstd, and xz compressed archives. Only BSD tar supports lz4 natively (but GNU tar can do an equivalent with --use-compress-program=lz4/-Ilz4). When creating archives both support the -a switch to automatically filter the created archive through the right compression program based on the file extension. While BSD tar recognizes compression formats based on the format, GNU tar only guesses based on the file extension.

See also #Archiving only usage.

Compression tools

Compression only

These compression programs implement their own file format.

Name	Package	Manual	Ext	Tar ext	Description	Parallel implementations
bzip2	bzip2	bzip2(1)	.bz2, .bz	.tbz2, .tbz	Uses the Burrows–Wheeler algorithm.	lbzip2, pbzip2
bzip3	bzip3	bzip3(1)	.bz3	.tbz3	Uses the Burrows–Wheeler algorithm.
gzip	gzip	gzip(1)	.gz, .z	.tgz, .taz	GNU zip, based on DEFLATE algorithm.	pigz, bgzip(1) (part of htslib^AUR), crabz^AUR, python-rapidgzip^AUR
lrzip	lrzip	lrzip(1)	.lrz		Improved version of rzip, uses multiple algorithms.	Is multithreaded by default
LZ4	lz4	lz4(1)	.lz4		Written in C, focused on compression and decompression speed.	Is multithreaded by default. See https://lz4.org/ for alternatives.
lzip	lzip	lzip(1)	.lz		Uses LZMA.	plzip^AUR
lzop	lzop	lzop(1)	.lzo	.tzo	Uses the LZO library (lzo).
xz	xz	xz(1)	.xz, .lzma	.txz, .tlz	Uses LZMA2. Default for GNU coreutils and kernel archive files.	Is multithreaded by default. An alternative is pixz.
zstd	zstd	zstd(1)	.zst		Uses Zstandard algorithm.	is multithreaded

Parallel implementations offer improved speeds by using multiple CPU cores.
Tar extensions refers to compressed archives where tar and the compression tool is used, e.g. .tzo is .tar.lzo.
See also #Compression only usage.

Archiving and compression

Name	Packages	Manuals	Ext	Description
p7zip	p7zip	7z(1)	.7z	The third-party POSIX port of 7-zip's command-line.
7-Zip	7-zip^AUR, 7-zip-full^AUR	Official manual	.7z	The upstream Linux version of a file archiver with a high compression ratio.
DAR	dar^AUR	dar(1)	.dar	Archiver to backup large live filesystems, takes care of hard links, extended attributes, sparse files and inode types.
RAR	rar^AUR, unrar	rar(1)	.rar	Both the format and the rar utility are proprietary.
t2sz	t2sz^AUR		.tar.zst .tzst	Tar archiving utility in C with member-aligned zstd-compression
tarlz	tarlz^AUR	tarlz(1)	.tar.lz .tlz	Tar archiving utility in C++ with member-aligned lzip compression
ZIP	zip, unzip	zip(1), unzip(1)	.zip	Widely used outside of the Linux world.
Unarchiver	unarchiver	unar(1), lsar(1)	many	Command-line tool of a Mac application, supports over 40 archive formats.
ZPAQ	zpaq^AUR	zpaq(1)	.zpaq	A high compression ratio archiver written in C++, uses several algorithms.
LHa	lhasa, lha^AUR	lha(1)	.lzh (on Amiga: .lha)	LZH/LHA archiver, supports the lh7-method.
WinAce	unace	unace(1)	.ace	Both the ACE file format and the archiving tool are proprietary.

See also #Archiving and compression usage.

Feature charts

Some of the tools above are capable of handling multiple formats, allowing for fewer installed packages.

Decompress

Name	File					Archive
Name	gzip	bzip2	LZMA	xz	zstd	ZIP	RAR	7z	CAB
gzip	Yes	No	No	No	No	Partial¹	No	No	No
p7zip	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
7-zip^AUR, 7-zip-full^AUR	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
unarchiver	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes
zstd	Yes	No	Yes	Yes	Yes	No	No	No	No

gzip's gunzip can only decompress single member ZIP files.

Usage comparison

Archiving only usage

Name	Create archive	Extract archive	List content
tar(1)	`tar cfv archive.tar file1 file2`	`tar xfv archive.tar`	`tar -tvf archive.tar`
cpio(1)	`ls file1 file2 \| cpio -o > archive.cpio`	`cpio -i -vd < archive.cpio`	`cpio -t < archive.cpio`

Compression only usage

Name	Compress	Decompress	Decompress to stdout
bzip2(1)	`bzip2 file`	`bzip2 -d file.bz2`	`bzcat file.bz2`
gzip(1)	`gzip file`	`gzip -d file.gz`	`zcat file.gz`
lrzip(1)	`lrzip file` `lrztar folder`	`lrzip -d file.lrz` `lrztar -d folder.tar.lrz`	`lrzcat file.lrz`
lz4(1)	`lz4 file`	`lz4 file.lz4`	`lz4cat file.lz4`
xz(1)	`xz file`	`xz -d file.xz`	`xzcat file.xz`
zstd(1)	`zstd file`	`zstd -d file.zst`	`zstdcat file.zst`

Archiving and compression usage

Name	Compress	Decompress	Decompress to stdout	List content
7z(1)	`7z a archive.7z file1 file2`	`7z x archive.7z`	`7z e -so archive.7z file1`	`7z l archive.7z`
dar(1)	`dar -c archive -g file1 -g file2`	`dar -x archive`		`dar -l archive`
rar(1)	`rar a archive.rar file1 file2`	`rar x archive.rar`	`rar p -inul archive.rar file1`	`rar l archive.rar`
zip(1), unzip(1)	`zip archive.zip file1 file2`	`unzip archive.zip`	`unzip -p archive.zip file1`	`unzip -l archive.zip`
lha(1)	`lha ao7 archive.lzh file1 file2`	`lha x archive.lzh`		minimal: `lha l archive.lzh` verbose: `lha v archive.lzh`

Convenience tools

atool — Script for managing file archives of various types.

https://www.nongnu.org/atool/ || atool

dtrx — An intelligent archive extraction tool.

https://github.com/brettcs/dtrx || dtrx^AUR

J7Z — GUI for Linux in java which attempts to simplify data compression and backup. It can create 7z, BZip2, Zip, GZip, Tar archives.

http://j7z.xavion.name || j7z^AUR

ouch — A command line utility for easily compressing and decompressing files and directories

https://github.com/ouch-org/ouch || ouch

python-unp — Command line tool that can unpack archives easily.

https://github.com/mitsuhiko/unp || python-unp^AUR

unp — A script for unpacking a wide variety of archive formats.

https://tracker.debian.org/pkg/unp || unp

unpack — Wrapper script for handling multiple archive formats.

https://github.com/githaff/unpack || unpack-git^AUR

patool — Allows various archive types to be created, extracted, tested, listed, compared, searched and repacked.

https://wummel.github.io/patool/ || patool^AUR

Bash/Functions#Extract

Determining archive format

To extract an archive, its file format needs to be determined. If the file is properly named you can deduce its format from the file extension.

Otherwise you can use the file tool, see file(1).

Esoteric, rare or deprecated tools

Name	Packages	Ext	Description
ARC	arc^AUR	.arc, .ark	Was very popular during the early days of the dial-up BBS. Superseded by ZIP.
ARJ	arj	.arj	An archiver used on DOS/Windows in mid-1990s. This is an open source clone.
Cabinet	cabextract, unshield	.cab, .exe	A variety of installation technologies in Windows use the CAB format.
compress	ncompress	.Z	The de facto standard UNIX compression utility to success the Huffman-based pack(1) before gzip become a thing.
Inno Setup	innoextract	.exe	Installers created by Inno Setup.
PAR2	par2cmdline	.par2	Parity archiver for increased data integrity. See also Parchive.
shar	sharutils	.shar	Creates self-extracting archives that are valid shell scripts.
Zoo	zoo^AUR	.zoo	Was mostly popular on the OpenVMS operating system before PKZIP became popular.

File system compression

Some file systems support on-the-fly compression of file data:

Btrfs can be configured to compress individual files, directories, or entire volumes by default.
On ZFS, compression can be enabled on pools or file systems.

Device mapper compression

This article or section is out of date.

Reason: kvdo was dropped from AUR as it has been upstreamed in kernel 6.9. (Discuss in Talk:Archiving and compression)

There is work being done to mainline (integrate into the Linux kernel project) the open-sourced VDO project, which provides a deduplication and compression device mapper layer in the interest of increasing storage efficiency. The following packages are available:

vdo — Userspace tools for managing VDO volumes

https://github.com/dm-vdo/vdo || vdo^AUR

kvdo — A pair of kernel modules which provide pools of deduplicated and/or compressed block storage

https://github.com/dm-vdo/kvdo || kvdo-dkms^AUR^{[broken link: package not found]}

Compression libraries

Brotli — Compression algorithm for data streams using the LZ77 algorithm, Huffman coding and 2nd order context modeling.

https://github.com/google/brotli || brotli

libzip — Provides creation and extraction of ZIP files. Used by KDE and Deepin in place of the zip/unzip tools.

https://libzip.org || libzip

zlib — Compression library implementing the deflate compression method found in gzip and PKZIP.

https://www.zlib.net/ || zlib

Zopfli — High compress ratio file compressor from Google, using a deflate-compatible algorithm called zopfli.

https://github.com/google/zopfli || zopfli-git^AUR

Troubleshooting

Garbled file names when extracted

See Character encoding#Troubleshooting.

See also

Retrieved from "https://wiki.archlinux.org/index.php?title=Archiving_and_compression&oldid=818727"

Hidden categories: