This reverts commit e2346a9e since it
was supposed to go through review first.

The idea of this is to allow us to graph the number of packages per repository in grafana just like
we're already doing in the archive exporter.

Encrypted control rooms break the reaction-based interface.

Killing a build when it runs out of memory is better then having the
whole machine trash.

This version contains a fix for sourceballs.

Levente Polyak authored 7 months ago and Christian Heusel committed 2 months ago

We currently estimate up to 2G per day. This should fit well within our
current storage limits.

Levente Polyak authored 7 months ago and Sven-Hendrik Haase committed 2 months ago

We do this in other services and features, lets be consistent and
opt-put of usage reporting everywhere.

They haven't been used for over 7 years[1] and was just removed from
dbscripts[2][3].

[1] bcd20f65 ("Disable the integrity check timer and service.")
[2] dbscripts!58
[3] f354ae35 ("dbscripts: Bump to latest commit for repos/archive split fixes")

0ebae15 ("chore(cron-jobs): Remove obsolete integrity-check script and friends")
2edd43d ("feat(db-archive): Archive package files atomically")
a56c710 ("feat(db-functions): Check the archive index in check_reproducible")
ea98599 ("fix(db-functions): Suppress realpath error output")

Fix #531

With the dbscripts role no longer deployed on gemini, we need to deploy
rsyncd from another role, which this commit does. The rsync endpoint is
only accessible over our internal wireguard network, to reduce exposure
and we also get encryption.

Relates to #531

archivetools will add missing packages to the "packages" directory in
the archive[1], but it is only done once a day, so if a package is
updated more than once in that timeframe, the earlier version will not
be added to the archive.

For this reason, we need to sync archived packages from repos.a.o to
gemini.a.o, so we don't lose packages, but also because we want the
archived packages to show up faster on the archive server than once a
day.

The package files could already exist on gemini, created by
archivetools, in which case rsync just ignores the files
(--ignore-existing).

The archived package files are deleted on repos.a.o after three days, so
there is some leeway to stop archive-cleanup.timer if the sync breaks
for whatever reason.

[1] https://gitlab.archlinux.org/archlinux/archivetools/-/blob/d5cbd5a5a9454eecd5c46b64ebc08d5087a25f2d/archive.sh#L141-L200

Relates to #531

We want to have the repos and the archive on different servers, to
ensure separation of concerns.

The I/O on gemini is also very sluggish, which is likely related to the
archive design (millions of files on spinning rust), and this has
resulted in rsync deleting the package database files due to a race
condtion[1][2]. The does not solve the race condition, but it should
make it less likely.

We also need to rethink the archive design, but for now we are just
isolating the problem.

There is some "brokenness" related to the archive no longer being on the
same server as the repos, which will be fixed in subsequent commits.

[1] https://lore.kernel.org/linux-btrfs/00ed09b9-d60c-4605-b3b6-f4e79bf92fca@foutras.com/
[2] https://lore.kernel.org/linux-btrfs/ZP8AWKMVYOY0mAwq@debian0.Home/

Relates to #531

The last bump was to a commit signed by him.

Fixes: 28f8ba48 ("dbscripts: Upgrade to latest commit")

Relates to: 701c1d01

/srv/repos/lock is the lock file location configured via a dbscripts
config. Only accessible for users in junior-packager group, a lot better
would be if sourceballs could write a lock file into
/run/sourceballs.lck as it is unrelated to /srv/repos.

Since 257 DynamicUser sets PrivateTmp=disconnected making debuginfod
unable to read/write to /var/tmp/ properly  hampering debuginfod's
functioning.

You can't `systemctl reload debuginfod` after installing the systemd
unit we need a daemon-reload.

This was apparently hosted on the long gone "apollo" server[1], and when
archweb was migrated to a dedicated cloud VM, it was changed to a
redirect to the main site (archlinux.org)[2][3].

It may have made sense at the time, but now four years later there is no
reason for keeping this around.

I guess dev.archlinux.org was something similar to what pkgbuild.com is
today ("Public HTML server" for staff), but only for developers.

[1] f6c3af0e ("Merge branch 'apollo_decomission' into 'master'")
[2] 824fb084 ("tf-stage1/archlinux: Change DNS records for the archweb migration and also increase the machine size")
[3] 9800d023 ("roles/archweb: Create domain redirects for the domains that point to specific archweb sub urls.")

Somehow these changes were not directly applied even though the role
reloads the prometheus config.

Fixes: 10475a62 ("prometheus: Alert if a build hosts is OOM for 12h")
Signed-off-by: Christian Heusel <christian@heusel.eu>

Fixes: 4159a61f ("dbscripts: switch to Git packaging")

Signed-off-by: Christian Heusel <christian@heusel.eu>

There is not much value in knowing when one of our build hosts has no
more memory left as all of them have plenty of swap available.
Additionally these rules trigger quite often even for short spikes.

Signed-off-by: Christian Heusel <christian@heusel.eu>

Link: https://lists.archlinux.org/archives/list/staff@lists.archlinux.org/message/3JNJQJ7O3LBIXST2EFBCAKM2HCHXJZUM/


Signed-off-by: Christian Heusel <christian@heusel.eu>

Signed-off-by: Christian Heusel <christian@heusel.eu>

This was added more than 7 years ago[1] and has not been relevant for a
long time.

[1] d32ce421 ("postfix: Remove compat_maps")