The sequoia cli once again changed their interface, therefore port this
to the (somewhat guaranteed to be) stable sops interface.

Suggested-by: David Runge <dvzrv@archlinux.org>
Signed-off-by: Christian Heusel <christian@heusel.eu>

aurweb: Skip commit validation if we are not on prod

See merge request !916

The sandbox deployments do not have access to the vault and so we can't have them use
it.

It will fail since we do not connect aur-dev instances to monitoring.

This makes it so that you don't need to be in the list of trusted GPG
keys `aurweb_pgp_keys` in order to start a test instance for the AUR. I
think `aurweb_pgp_keys` should only be for people wanting to deploy this
to production.

It was installed for checkservices which we rarely use anymore due to
automated updates. Also, update README.md to remove references to it.

This reverts commit e2346a9e since it
was supposed to go through review first.

The idea of this is to allow us to graph the number of packages per repository in grafana just like
we're already doing in the archive exporter.

Encrypted control rooms break the reaction-based interface.

archusers: restrict jouke user to mail.archlinux.org

See merge request !919

Killing a build when it runs out of memory is better then having the
whole machine trash.

This version contains a fix for sourceballs.

The last gitlab bot[1] for the aurweb-tfstate project expired[2] some
time ago, but it was not replaced with a new project bot.

A new project bot has just been added, because we need the use-case[3]
working again.

[1] 7a994067 ("gluebuddy: Add new gitlab bot for the aurweb-tfstate project")
[2] 242213b2 ("gluebuddy: Remove deleted project bot for arch-boxes")
[3] 5fb8df85 ("gluebuddy: Add gitlab bot for aurweb-tfstate project")

tempo: increase retention to 7 days

See merge request !914

Levente Polyak authored 6 months ago and Christian Heusel committed 1 month ago

We currently estimate up to 2G per day. This should fit well within our
current storage limits.

one-shots: Remove obsolete keycloak-importer

See merge request !903

Kristian Klausen authored 2 months ago and Christian Heusel committed 1 month ago

It has not been used for over 4.5 years and I think it is unlikely that
we will need it in the future.

tempo: Clean up config a bit

See merge request !915

Levente Polyak authored 6 months ago and Sven-Hendrik Haase committed 1 month ago

We do this in other services and features, lets be consistent and
opt-put of usage reporting everywhere.

Gemini doesn't need to send emails and hasn't had that need for several
years (AFAIU). It may have been used by the integrity-check script in
the past, but that was disabled 7 years ago[1] and was just removed
completely from dbscripts[2][3] and the infra repo[4].

[1] bcd20f65 ("Disable the integrity check timer and service.")
[2] dbscripts!58
[3] f354ae35 ("dbscripts: Bump to latest commit for repos/archive split fixes")
[4] 8e22cdbe ("dbscripts: Remove obsolete integrity-check.{service,timer}")

They haven't been used for over 7 years[1] and was just removed from
dbscripts[2][3].

[1] bcd20f65 ("Disable the integrity check timer and service.")
[2] dbscripts!58
[3] f354ae35 ("dbscripts: Bump to latest commit for repos/archive split fixes")

Move the repos and associated services to its own server

Closes #531

See merge request !905

0ebae15 ("chore(cron-jobs): Remove obsolete integrity-check script and friends")
2edd43d ("feat(db-archive): Archive package files atomically")
a56c710 ("feat(db-functions): Check the archive index in check_reproducible")
ea98599 ("fix(db-functions): Suppress realpath error output")

Fix #531

With the dbscripts role no longer deployed on gemini, we need to deploy
rsyncd from another role, which this commit does. The rsync endpoint is
only accessible over our internal wireguard network, to reduce exposure
and we also get encryption.

Relates to #531

archivetools will add missing packages to the "packages" directory in
the archive[1], but it is only done once a day, so if a package is
updated more than once in that timeframe, the earlier version will not
be added to the archive.

For this reason, we need to sync archived packages from repos.a.o to
gemini.a.o, so we don't lose packages, but also because we want the
archived packages to show up faster on the archive server than once a
day.

The package files could already exist on gemini, created by
archivetools, in which case rsync just ignores the files
(--ignore-existing).

The archived package files are deleted on repos.a.o after three days, so
there is some leeway to stop archive-cleanup.timer if the sync breaks
for whatever reason.

[1] https://gitlab.archlinux.org/archlinux/archivetools/-/blob/d5cbd5a5a9454eecd5c46b64ebc08d5087a25f2d/archive.sh#L141-L200

Relates to #531

We want to have the repos and the archive on different servers, to
ensure separation of concerns.

The I/O on gemini is also very sluggish, which is likely related to the
archive design (millions of files on spinning rust), and this has
resulted in rsync deleting the package database files due to a race
condtion[1][2]. The does not solve the race condition, but it should
make it less likely.

We also need to rethink the archive design, but for now we are just
isolating the problem.

There is some "brokenness" related to the archive no longer being on the
same server as the repos, which will be fixed in subsequent commits.

[1] https://lore.kernel.org/linux-btrfs/00ed09b9-d60c-4605-b3b6-f4e79bf92fca@foutras.com/
[2] https://lore.kernel.org/linux-btrfs/ZP8AWKMVYOY0mAwq@debian0.Home/

Relates to #531

The last bump was to a commit signed by him.

Fixes: 28f8ba48 ("dbscripts: Upgrade to latest commit")

Offboard coderobe as Package Maintainer

Closes #655

See merge request !912