http_requests_total  contains requests from debuginfo.al.org host
as well as from aur.al.org so filter them on job 'aurweb'

Signed-off-by: Leonidas Spyropoulos <artafinde@archlinux.org>

Using plain "borg" resolves to /usr/local/bin/borg which is the wrapper
for our main backup host. This causes the offsite backup to be executed
with BORG_REPO set to the main backup destination.

While the above doesn't cause any issues with the backup script/service,
because borg invocations specify the backup destination as an argument,
it's not ideal and/or correct. Adjust borg_cmd to include the full path
of /usr/bin/borg, thus removing any ambiguity.

rsync.net has upgraded to borg 1.2 and we can now run borg compact on
both rsync.net and the Hetzner storage box.

Fixes: 17927c9a ("borg_client: run compact after pruning on borg 1.2")

Update /etc/rebuilderd-sync.conf to use the Geo host mirror instead of
europe.mirror.pkgbuild.com (for added reliability in case the latter's
availability is impacted).

The /api/v0/build/report endpoint has received POSTs up to 161M so far
this year (2022). In 2021 there had been POSTs of sizes up to 404M and
up to 814M for 2020. Multiple hundreds of MB seem a bit excessive, but
we should be able to do up to 200M.

Remove a few stray TABs and correct double-indented lines.

Upstream archlinux-repro was already using europe.mirror.pkgbuild.com as
its bootstrap and regular mirror. Furthermore, since [1] it has switched
to the Geo mirror. Remove both vars from /etc/archlinux-repro/repro.conf
and use the default mirror values (which are more than suitable).

[1] https://github.com/archlinux/archlinux-repro/commit/c024b892d07a

Fixes: 68ec7871 ("aurweb: Mirror aur.git to GitHub[1]")

The burst size of 300 reportedly allows ~150 git operations. This might
not always be sufficient when installing a lot of packages from the AUR.

Specify a higher burst size to cover most legit use cases, even if this
makes us more susceptible to abuse.

"Disabling revoked keys in keyring" when running "pacman-key --populate"
is very slow (easily +20 seconds), in our case the boot is now ~27
seconds faster (tested on secure-runner1). The pacman master private key
is removed to prevent malicious actors from injecting packages, a new
key is generated by pacman-init.service on boot.

[1] https://archlinux.org/news/qemu-700-changes-split-package-setup/

Changes:
- Switch to arch-boxes' base image
- Verify the base image's signature
- Use the new "latest" symlink, instead of parsing the HTML for
  finding the latest arch-boxes image[1]
- Create the base image by using arch-chroot and friends, instead of
  creating a full-blown VM
- Create the VMs from domain XML template instead of virt-clone
- Switch mirror to geo.mirror.pkgbuild.com
- Try to follow "filesystem hierarchy" standards for where to place
  configuration (id_ed25519) and "vendor data" (arch-boxes.asc and
  domain_template.xml)
- Use a ed25519 key instead of RSA key
- Only start the "update base image" server if network and DNS are up
- Misc fixes and cleanups

[1] !552

Upstream now provides a solution for setting the "staging dir" for
fastzip[1].

[1] https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3130

For some workloads running in a container is too restrictive, ex:
arch-boxes (loop device, filesystem mount, pacstrap) and archiso
(pacstrap). Currently they both run a TCG accelerated QEMU VM, which is
very slow and painful to work with. We should provide a better option to
our users!

This adds a hardware accelerated VM for this kinds of workloads, which
is way faster and you can do whatever you like (mostly)!

Fix #283

The service hardening options have been included in geoipupdate 4.9.0-3.

[1] https://bugs.archlinux.org/task/75434

This is meant as a extra "backup" and as another way for our users to
fetch PKGBUILDs from the AUR. It also allows the community to create
their own (perhaps better) "AUR" API/database as all essential data is
now available (this + [2]).

At the monent this is experimental and we aren't committing to keeping
it around.

[1] https://github.com/archlinux/aur
[2] http://aur.archlinux.org/packages-meta-ext-v1.json.gz

This allows to retain all default hardening options that memcached@.service has.

The glibc 2.35-6 package ships with the C.UTF-8 locale included which
means there is now a usable UTF-8 locale available by default.

en_US.UTF-8 will still be generated because PostgreSQL clusters are
created with that locale. Migrating the clusters to C.UTF-8 is
possible, but that requires dumping and recreating them.

* Remove www. from archlinux.org,
* Use HTTPS for the license link,
* Update $wgGitRepositoryViewers,
* Update comments referencing paths and URLs.

After starting a new container with the latest version of GitLab, opt to
remove older docker images so they do not take up disk space needlessly.

Also tweak the documentation on rebuilderd workers and add runner1.

The upstream `sample_config.yaml` is now minimal, so we cannot diff with
it anymore.

Adding/removing pages from the whitelist when using the Vector skin requires the write API now.
Without this, regular users get this error:

    You're not allowed to edit this wiki through the API.

Jakub Klinkovský authored 2 years ago and Kristian Klausen committed 2 years ago

- $wgScriptExtension was removed in MW 1.31
- $wgDBmysql5 was removed in MW 1.33
- $wgShowIPinHeader was removed in MW 1.27
- $wgUseETag was removed in MW 1.28
- $wgEnableWriteAPI was removed in MW 1.32
- $wgShowSQLErrors was removed in MW 1.37

Jakub Klinkovský authored 2 years ago and Kristian Klausen committed 2 years ago

$wgFragmentMode = [ 'html5', 'legacy' ]; is the default value since
MediaWiki 1.37:
https://www.mediawiki.org/wiki/Special:MyLanguage/Manual:$wgFragmentMode

Jakub Klinkovský authored 2 years ago and Kristian Klausen committed 2 years ago

These were fixed in MediaWiki 1.38

Jakub Klinkovský authored 2 years ago and Kristian Klausen committed 2 years ago

Also updated some settings:

- The $wgShellLocale setting was removed from MediaWiki, see
  https://www.mediawiki.org/wiki/Special:MyLanguage/MediaWiki_1.38
- The "new" Vector skin got a name "vector-2022" and can be set simply
  by $wgDefaultSkin = 'vector-2022'; see
  https://www.mediawiki.org/wiki/Skin:Vector/2022#Note_about_1.38_release

Allow only .sig, .torrent and .txt.

This is done to prevent downloading files such as https://archlinux.org/iso/latest/arch/boot/x86_64/vmlinuz-linux.

Since release 2022.07.01, there is a versionless bootstrap tarball file (archlinux-bootstrap-x86_64.tar.gz). See https://github.com/pierres/archiso-manager/pull/12.

"latest" is a valid version, so there is no need to check for the exact version number anymore.

This restores public access to recent changes, page history etc. which
was disabled in September 2019 due to misbehaving crawlers - see commits
https://github.com/archlinux/archwiki/commit/f2c518d3df94e9ad6898fdd03cac982902f8ac36
and https://github.com/archlinux/archwiki/commit/8595e8d66141e608db794f1deef3e52cb36e5e78.

Let's hope that our robots.txt has been improved since then and the
server load will not increase much. If it proves to be still a problem,
this commit can be reverted and we will figure out better solution.