Skip to content

Change explicit default for compression level of zstd

Kristian Klausen requested to merge github/fork/cherti/master into master

Created by: cherti

This PR removes the explicit default of -15 for zstd compression.

The reason for this proposal is, as can be seen in the plot below, that -15 is still paying significant CPU cost for barely any gain. By removing -15, zstd would, if the user didn't specify anything else, resort to level 3, which saves significant compression times while raising the filesize only by 2.5% of the original size.

Furthermore, this would make the behavior of mkinitcpio less surprising in this regard, as all other compressors are used with their default levels, only zstd has an explicit level set, potentially leading to user confusion, why zstd, known among other things for its compression speed, is slower than other compressors like lz4.

Given the data shown below and the principle of least surprise, and given that the default of zstd already performs equal in size to lz4 but noticably better in compression ratio, I think that -15 should be lowered for the default choice and not explicitly setting a compression level (and hence using zstd -3) is a sane default here.


Notation: compression-algorithm.level

Methodology: The displayed values are median values of multiple runs, to reduce statistical flaking, the original sample was cached from disk to reduce the impact of disk-delay, and the system was run with only the Terminal open holding the benchmark process, to reduce noise from other applications. Standard background processes were not stopped, interference of those is considered to be dealt with by the multiple runs and median-taking afterwards.

Decompression remark: For decompression the level doesn't matter, zstd is so fast that I can't even say that the decompression times I get are decompression times and not dominated by things that are not zstd actually doing work (for all levels we are solidly below 0.1s for decompression time, and the decompression time is barely even systematically dependent on the compression level, so when thinking about the desired level, decompression time doesn't seem to bring anything to the table it terms of reasons for decisions).

Merge request reports