Enabling debug packages by default has resulted in more debug packages.

Nobody could have foreseen this.

When debuginfod processes big debug packages it can run out of memory
and get killed. Then, it gets restarted by systemd only to OOM again.

Let's try doubling the memory to ~4G.

Equinix's AMS1 DC is being shut down so we need to recreate this box.

For Geo variety, this one is created in Frankfurt instead of Amsterdam.

Ref #495

Equinix's AMS1 DC is being shut down so we need to recreate this box.

Ref #495

We've been seeing CPU spikes that a single vCPU can't handle, resulting
in alerts about high CPU usage. Let's try two cores to see how they do.

For the upcoming "Git packaging sources"[1], we need a shadow copy of
repos.archlinux.org we can test stuff on (Ansible roles, packaging
workflow etc.). We still have repos-git.archlinux.org, but that box is
more of a POC box allocated to anthraxx for his MRs related to the git
migration.

More details on how the box was setup can be found in[2]. Only the
terraform code is committed to the master branch, as it is annoying
having the server in the state without the corresponding IaC code
comitted.

This was deployed Fri Dec 16 01:10:15 2022 +0100.

[1] https://lists.archlinux.org/archives/list/arch-dev-public@lists.archlinux.org/thread/YKKYBXQL62U5RTYIRI2NT2I3EG7V63HT/
[2] !665

Fix #479

As announced on the mailing list[2] pacman has been migrated to gitlab
and there is no real use for patchwork left, so it can be
decommissioned. A static copy[1] is kept around for the time being to
avoid link rot.

[1] https://gitlab.archlinux.org/archlinux/patchwork-archive
[2] https://lists.archlinux.org/archives/list/arch-dev-public@lists.archlinux.org/message/7B6R5HVEC67U7B2VQ3SKUVXU4RDCRRMM/

Fix #487

We currently have 23G of debug packages and soon expect quite a bit more
since the debug option is being enabled by default in devtools 20230105.

Ref: rfcs!15

- hcloud: 1.34.3 -> 1.35.2
- hetznerdns: 2.1.0 -> 2.2.0

With the final lists migrated to mailman3[1], the mailman2 server can
finally be killed.

When the mailman3 server was initially setup[2], it was done on a
separate server because the mailman and mailman3 packages conflicted,
and the traffic was routed over wireguard (HTTP, LMTP and SMTP).

Instead of installing mailman3 on the original lists.al.org server and
transferring the data, it was easier just to install the missing pieces
(basically Postfix and adjusting the Nginx configuration) on the ml3
server and move the IPs (to keep the IP mail reputation).

So basically the following was done:
- The IPs for the original lists.al.org was moved to the mailman3.al.org
  server
- The mailman2 datadir was transferred to mailman3.al.org server, so we
  can keep the pipermail links alive, and import missing mails if needed
- The original lists.al.org server was decommissioned
- The mailman3.al.org server was renamed to lists.al.org
- The missing pieces was added to the mailman3 role (basically Postfix +
  Nginx adjustments)
- The mailman role was deleted and the mailman3 role renamed to mailman

[1] 75ac7d09 ("mailman: Fourth and final batch of mailman3 migrated lists")
[2] 9294828f ("Setup mailman3 server")

Fix #59

Going to be served by all our Geo boxes under riscv.mirror.pkgbuild.com.

The WKD logic has been moved to the archlinux-keyring project[1][2].

[1] archlinux/archlinux-keyring!166
[2] archlinux/archlinux-keyring!169

Useful if we wanted to create a Geo-based archive consisting of machines
in the archive_mirrors group (though this will likely not happen because
it'd break archlinux-repro due to the ~4 hour sync delay).

aur4 was apparently some kind of "new aur" hosted on luna, which is long
gone.

Fixes: 79f7d599 ("Goodbye luna")

anthraxx wants some infra for a repos-git POC, so let's give it to him!

The server has been configured with the common and firewalld role, but
is unmanaged.

Its disks were migrated to a new server (prompted by an unsolvable issue
with the previous box's network interface; might have been a mobo issue).

Brings support for managing Primary IPs.

There's a bit of a chicken and egg situation here but it's preferable to
manage the server's attributes the same way as all of the cloud servers.

Ref: archlinux/monthly-reports!1

Ref: repod!65

We want non-DevOps to be able to deploy project documentation (ex:
repod) with GitLab Pages and a separate domain was considered the only
sensible solution due to security issues[1].

[1] https://github.blog/2013-04-09-yummy-cookies-across-domains/

asia.mirror.pkgbuild.com has been offline for 12 days so far while we
wait for a NIC replacement. Should have taken it out of DNS NS duties
earlier but better late than never.

It needs the extra RAM.

[1] https://datatracker.ietf.org/doc/html/rfc8461#section-3.2

Fixes: 0b87cbfd ("mta_sts: Switch to enforce mode and bump max_age to 30 days")

Ansible side of commit 5007c1a8 ("tf-stage1: allow setting the NS
TTL of geo domains"); both values need to match so our geo nameservers
report the same TTL as that returned by the parent zone's nameservers.

When adding a new geo domain or doing other testing, we would want to
use a low TTL to allow for making quick changes to the configuration.

- add the new role to redirect.archlinux.org
- release mirror.pkgbuild.com of all DNS duties

Foxboron wants some infra for a buildbot POC, so let's give it to him!

The server has been configured with the common and firewalld role.

The idea bebind this is to be able to give vault access to new DevOps
members without giving away more important credentials like Hetzner's.

In an effort to stay consistent with the TTL used for the archlinux.org
and pkgbuild.com NS records, as well as slightly improve lookup latency.

New hcloud adds protection fields to servers, volumes and floating IPs.

/srv/gitlab has been moved to local (NVMe SSD) storage; hopefully it
won't grow too large and thus require transferring back to a volume.

We don't want mirror.pkgbuild.com's DNS server to be a
single-point-of-failure, so this commit adds multiple authoritative DNS
servers for the zone. The extra DNS servers are run on the geomirror
servers.

The _acme-challenge zone, used for obtaining certificates, is run solely
on mirror.pkgbuild.com's DNS server, to avoid syncing DNS records
between the servers (KISS).