Skip to content

functions: performance boost with tar usage

Tobias Powalowski requested to merge tpowa/mkinitcpio:tpowa-use-tar into master

Why this merge request?
The used fallback routine, which is enabled by default in mkinitcpio setup is pretty slow and can be improved.
With autodetection disabled it needs quite long to collect all files. The changes improve the situation by around 10-20%.

How to test?
Only build default fallback (autodetect is skipped!):
HOOKS=(base udev modconf kms keyboard keymap consolefont block filesystems fsck)

Why using tar and not bsdtar?
Reason: tar is faster than bsdtar while copying files.
tar -C / -cpf - 'directory' | tar -C 'destdir' -xpf -
vs.
bsdtar -C / -cpf - 'directory' | bsdtar -C 'destdir' -xpf -
My measurements show around 20% faster (diskio is not relevant, tests were made on ramfs drive)

Why replacing the whole add_file routine?

  • add_file routine with faster tar/chmod routine
    tar can handle all types of files (directory/symlink/file/replacement) including special permissions.
    With this routine you can reduce the actions, that are taken now to a minimum.
    User set permissions are probably rare and chmod will not be called that often.

Why copying all modules in one call?

  • install all modules at once:
    With copying all collected modules in one call with tar is faster, than spanning a process for each file in the for loop.

Why add a switch to add_full_dir:

  • add_full_dir with tar usage switch
    Fixes BUG: The add_full_dir routine does not handle hidden dot files.
    The routine which is used now is quite complex. Simply copying can be done easier and faster with one tar call while keeping the old functions.
Edited by Tobias Powalowski

Merge request reports