Draft: let makepkg manage sources
This MR proposes an alternative way to manage the git repositories required to build electron.
The current approach is letting makepkg manage the largest repo, chromium-mirror(or src
, 43GB). Then in prepare()
, gclient
pulls other git repos(approx 170 repos, 39GB) from Internet. This still generates a lot of network traffic and it happens every time when building electron package.
To further reduce the network traffic for building electron and use the idiomatic way to manage package source code. I think it's a good idea to let makepkg manage all source repos. This PR comes up a script electron-roller.py
, which is a standard-alone, simplified version of DEPS file parser. It could recursively parse the DEPS file and generate the list of urls for source
in PKGBUILD and script to put together the repos to make it ready for build.
There's another kind of dependency that is used during the build, the cipd dependencies. They are usually prebuilt toolchains like ninja, gn, esbuild and jdk. The PKGBUILD is already trying to use the packaged toolchains. This script takes it a step further to avoid downloading the prebuilt toolchains from google, further reducing network traffic. The script only enables whitelisted cipd dependencies that are really necessary for build: src/third_party/screen-ai/linux
and esbuild
(The one from extra repo cannot be used because the version is incompatible).
Benefits
- Reduce a lot of network traffic when building electron
- makepkg is better at caching these repos. And they could be shared by multiple versions of electrons via
SRCDEST
.
- makepkg is better at caching these repos. And they could be shared by multiple versions of electrons via
- Doesn't download prebuilt binaries from google except whitelisted cipd dependencies.
Down Sides
- The PKGBUILD will contain many lines of code generated by
electron-roller.py
, making it a bit uncomfortable to read. - The initial build without any repos cloned by makepkg will be slower than before, because makepkg clones the repos in a single-threaded manner, slower than multi-threaded
gclient
.
Maintenance
To update to a new electron version, for example, run electron-roller.py update v29.1.0 electron29
. Then modify the pkgver in PKGBUILD as usual. electron-roller.py
itself shouldn't need to be updated in the future because electron pins the chromium major version so that the format of DEPS file is usually quite stable.
For electron major version update, electron-roller.py
might need to be tweaked to accommodate new necessary cipd dependencies and changes in the format of DEPS files. I am very happy to offer help if needed :)
Misc
This MR downgrades the version from 29.1.1 to 29.1.0 because I can't find the tag for 29.1.1. It seems that upstream deleted it.
This MR also removes 'is_official_build=false'
to fix the crash(#1 (closed)) and improve performance.
I would love to hear your opinions on this MR @alerque