Skip to content

Draft: let makepkg manage sources

Levi Zim requested to merge kxxt/electron29:makepkg-sources into main

This MR proposes an alternative way to manage the git repositories required to build electron.

The current approach is letting makepkg manage the largest repo, chromium-mirror(or src, 43GB). Then in prepare(), gclient pulls other git repos(approx 170 repos, 39GB) from Internet. This still generates a lot of network traffic and it happens every time when building electron package.

To further reduce the network traffic for building electron and use the idiomatic way to manage package source code. I think it's a good idea to let makepkg manage all source repos. This PR comes up a script electron-roller.py, which is a standard-alone, simplified version of DEPS file parser. It could recursively parse the DEPS file and generate the list of urls for source in PKGBUILD and script to put together the repos to make it ready for build.

There's another kind of dependency that is used during the build, the cipd dependencies. They are usually prebuilt toolchains like ninja, gn, esbuild and jdk. The PKGBUILD is already trying to use the packaged toolchains. This script takes it a step further to avoid downloading the prebuilt toolchains from google, further reducing network traffic. The script only enables whitelisted cipd dependencies that are really necessary for build: src/third_party/screen-ai/linux and esbuild(The one from extra repo cannot be used because the version is incompatible).

Benefits

  • Reduce a lot of network traffic when building electron
    • makepkg is better at caching these repos. And they could be shared by multiple versions of electrons via SRCDEST.
  • Doesn't download prebuilt binaries from google except whitelisted cipd dependencies.

Down Sides

  • The PKGBUILD will contain many lines of code generated by electron-roller.py, making it a bit uncomfortable to read.
  • The initial build without any repos cloned by makepkg will be slower than before, because makepkg clones the repos in a single-threaded manner, slower than multi-threaded gclient.

Maintenance

To update to a new electron version, for example, run electron-roller.py update v29.1.0 electron29. Then modify the pkgver in PKGBUILD as usual. electron-roller.py itself shouldn't need to be updated in the future because electron pins the chromium major version so that the format of DEPS file is usually quite stable.

For electron major version update, electron-roller.py might need to be tweaked to accommodate new necessary cipd dependencies and changes in the format of DEPS files. I am very happy to offer help if needed :)

Misc

This MR downgrades the version from 29.1.1 to 29.1.0 because I can't find the tag for 29.1.1. It seems that upstream deleted it. This MR also removes 'is_official_build=false' to fix the crash(#1 (closed)) and improve performance.

I would love to hear your opinions on this MR @alerque

Merge request reports