Use mold as default linker
While we can parallelize compile processes easily, the linker in use, namely bfd is currently single threaded. While gold improved this behavior, mold is by far the quickest [0].
But there are downsides as well: Mold supports only a small subset of linker scripts, also some packages hard depend on bfd and write their tests specifically for the behavior of bfd. This leads to a list of incompatible packages first shared with gentoo [1].
Due to these issues, replacing /usr/bin/ld has been discarded, instead
the required LDFLAGS have been added.
Broken packages can be worked around by setting
LDFLAGS="${LDFLAGS//fuse-ld=mold /}"
Go packages built with -extldflags=${LDFLAGS}
need to quote the flag
due to the introduced whitespace in LDFLAGS:
go build -ldflags "-v \"-extldflags=${LDFLAGS}\""
[0] https://github.com/rui314/mold/raw/main/docs/comparison.png [1] https://docs.google.com/spreadsheets/u/1/d/1LnyrUw79oY5DxBNfXGmlxr2-0n4V0AA06BzbO2cdhZg/htmlview#gid=0
Binaries can be checked for mold linking with readelf -p .comment <binary>
. Mold adds a line similar to
[ 0] mold 1.8.0 (a49a201695edd294ed4d97231c9dc5a994275dd2; compatible with GNU ld)
When this MR gets merged:
-
Add mold to [core] -
Add mold as dependency of devtools -
Add mold to group base-devel
Merge request reports
Activity
You may consider lld as well. Exclude
libLLVM-$ver.so
which is shared with other packages, the executable itself with libLLD* are small. FreeBSD and ChromeOS have proven that for many systems lld can link everything. It has a great linker script support.It is super robust: battle tested by FreeBSD, Chrome OS, Android, Chimera Linux, etc.
(If you comment out lld/ELF/LTO.cpp code and build it with
-DLLVM_LINK_LLVM_DYLIB=off
and-Wl,--gc-sections
, it will be very small as well.)The performance comparison on the mold website isn't really fair at least for newer versions of lld.
If you use mimalloc for both lld and mold, their performance gap should be very small for most packages. Only for some worse-case programs (heavy input file parsing), lld may take up to 1.5x mold time if there is no debug information (this is with 8 or more threads). -g or using fewer threads will dilute mold's advantage.
If there are Arch Linux specific packages, I suggest that you check with
-fuse-ld=lld -Wl,--warn-backrefs
, otherwise if a package gets adapted to lld or mold archive semantics, there may be a risk that the output may not be linkable with GNU ld or gold (https://maskray.me/blog/2021-06-13-dependency-related-linker-options#warn-backrefs).Edited by Fangrui SongThanks for the detailed explanation, and the linker performance comparison looks really promising.
Here are my two cents: As a binary distribution, we should be more focused on making the resulting binaries as optimal as possible, so aspects like resulting binary sizes or LTO improvement, etc may weigh more.
As far as I have discovered, ld.bfd still produces the smallest binaries, and mold will require GCC 13.1+ to fully support LTO: https://github.com/rui314/mold/issues/181#issuecomment-1266574026
Hmm, mold is super interesting...but this seems like quite a major change. Possibly should go through RFC first?
Seems like devs working on large code bases with rapid debug-edit-rebuild cycles would benefit the most. Have you done much testing @freswa? I'd be interested to know how many pkgs are likely to have issues and whether the build-time savings are worth it.
Some concerns off the top of my head (possibly unfounded):
- buy-in from other big distros
- bus factor seems pretty high
- licensing uncertainty
Edited by ToolybirdI compiled ~100 pkgs where I've encountered issues only on packages from the list I mentioned earlier. I didn't do much performance testing, since I'm using mold for some months now. The bigger the binary, the more advantage you gain with using mold. Especially with Rust it's a nice linker.
Considering the arguments from @maskray I'm rethinking about this MR.
@freswa Considering that a lot of Go packages would simply break I think we need to pro/con this a better than as a simple MR.
I think an RFC would make a lot more sense before we consider merging this
added scopeenhancement scopequestion statuson-hold labels