Talk:Archiving and compression

From ArchWiki
Latest comment: 1 hour ago by Hanabishi in topic Decompress formats

TODO: New parallel gzip decompression tool: pragzip/igzip

As on 2022, pigz(1) still does not support true parallel decompression of gzip stream. A recently implemented decompression-only Python/C++ gzip library (incl. cli utility in C++ called igzip) that is capable to decompress pigz/bgzip(1)'s output parallelly.

Impl details: https://github.com/madler/pigz/issues/36#issuecomment-1004248819

The implementation is nearly stable but no distro is packaging it per repology (the repository layout is a bit confusing too, shared by both ibzip and igzip and two library.).

As a pending backend of the already pkg'd ratarmountAUR, once it get pkg'd, should it be added to the table as this increased the performance on decompression of large gzip file significantly.

-- Jlhs (talk) 15:12, 13 September 2022 (UTC)Reply[reply]

Thank you for linking to my comment about indexed_bzip2 / rapidgzip on the pigz issue. Some short corrections: The Python/C++ utility for parallel decompression is called rapidgzip (at the point of the comment it was called pragzip). It also got its own repository by now even though it has still the same structure as indexed_bzip2. On the other hand, igzip is the command line utility of the ISA-L library, which is also a quite fast gzip decompressor, twice as fast as pigz and roughly 4x as fast as gzip on my system, and roughly as fast as libdeflate.

I think that rapidgzip is at a point at which I should spent some time trying to get it into repositories. I'll probably try a bit with AUR and then maybe even try with Debian. I never added a package to either, though.

--Mxmlnkn (talk) 19:51, 8 September 2023 (UTC)Reply[reply]

Decompress formats

The table in Archiving and compression#Decompress lists a lot of esoteric formats like LHa/LZH, compress and ARJ while missing, IMO, more common ones like LZMA, LZMA2, zstd and archives like 7z. I propose replacing these columns. See draft below. -- nl6720 (talk) 11:56, 28 March 2024 (UTC)Reply[reply]

Perhaps bzip2 could be removed or replaced with lz4? -- nl6720 (talk) 11:58, 28 March 2024 (UTC)Reply[reply]
I think gzip is not even worth mentioning here, "single member ZIP files" is not a real thing that people use.
Hanabishi (talk) 12:08, 28 March 2024 (UTC)Reply[reply]

Decompress (draft)

Type File Archive
Name gzip bzip2 LZMA xz zstd ZIP RAR 7z CAB
gzip Yes No No No No Partial1 No No No
p7zip Yes Yes Yes Yes Yes Yes Yes Yes Yes
7-zipAUR, 7-zip-fullAUR Yes Yes Yes Yes No Yes Yes Yes Yes
unarchiver Yes Yes Yes Yes No Yes Yes Yes Yes
zstd Yes No Yes Yes Yes No No No No
  1. gzip's gunzip can only decompress single member ZIP files.