CPU: Intel Xeon E5-2620 -> E-2288G
Disk: 2x~1TB -> 2x~500GB

Split storage box monitoring into new text collector

See merge request !466

This was previously monitored as part of the borg text collector, but
now that it only runs after each backup (instead of hourly) the stats
from monitoring.archlinux.org do not remain accurate for long. Switch
back to hourly checks of the storage box's disk usage by adding a new
text collector just for this purpose.

Run borg-textcollector after each backup completes

See merge request !465

Instead of gathering borg statistics every hour or so, run the text
collector script only once after each borg-backup service finishes.

Also split the borg text collector script into two similar scripts,
where each one gathers borg statistics for its respective borg host.

Use RandomizedDelaySec=30min in Borg TextCollector

See merge request !464

Doing this in an attempt to be kind to our Borg hosts in cases where the
prometheus-borg-textcollector.timer is restarted on all hosts and avoids
having all machines querying the Borg hosts within the same minute. Only
downside is that the timers will trigger every 75-ish minutes instead of
exactly every hour, but this should not be a problem.

Split the postfix role into a role for mail.a.o and the clients

See merge request !454

The role for the clients is named postfix_null (per [1]) and it's much
simpler and cleaner than the postfix role. I hope can cleanup the
postfix role at a later date.

[1] http://www.postfix.org/STANDARD_CONFIGURATION_README.html#null_client

Fixes: cf9c92fd ("dovecot: Disable POP3")

Implicit TLS is the future[1].

[1] https://datatracker.ietf.org/doc/html/rfc8314

No one uses it and less to worry about.

Fix #205

Restrict the mail users to passwd and decouple the mailboxes from the system user

See merge request !450

The homedir is now /home/vmail/%d/%n instead of /home/$USER.

Preparation for switching to a virtual user setup and removing all the
staff users from mail.a.o.

The users are only meant as a way to change the mail password and
setting up forwarding (~/.forward), the latter will be handled by the
DevOps team now.

Fixes: 678845af ("Add Kape server IPv6 addresses (fixes #230)")

Add redirects for git.archlinux.org using a map

See merge request !439

Increase zram-fraction to 1.0 for lists.archlinux.org

See merge request !460

It's been running out of swap during borg-backup and seems to get good
compression ratios; try upping the zram size to 100% of RAM (from 50%).

Add fail2ban exporter

See merge request !457

The fail2ban exporter exports the amount of bans per jail.

hcloud_inventory: Optimize --list to avoid --host calls

See merge request !459

By adding a top-level element called "_meta" to the --list response,
Ansible will not call the inventory script with --host for each host
thus saving a lot of time and many requests to the Hetzner Cloud API.

The speed-up is significant; `ansible-inventory --list` is down from
about 1 minute to just 7 seconds in my testing (with ~60ms latency).

As per the following deprecation warning (even though it has a typo):

[DEPRECATION WARNING]: [defaults]callback_whitelist option, normalizing
names to new standard, use callback_enabled instead. This feature will
be removed from ansible-core in version 2.15.  Deprecation warnings can
be disabled by setting deprecation_warnings=False in ansible.cfg.

[1] https://github.com/ansible/ansible/pull/74845

zswap seems like the better choice when a backing swap partition exists.

Remove zram size limit and disable zswap when using zram

See merge request !458

When both zswap and zram are active, zswap sits in front of zram and
treats it as a backing store. We just want to use zram and not zswap
disguising itself as such; disable the latter so we can enjoy useful
zramctl statistics.

Implemented as tmpfiles.d/zram.conf which disables zswap at runtime.

Restarting swap.target doesn't apply configuration changes; instead we
can restart systemd-zram-setup@zram0 which seems to do what we wanted.

Set "max-zram-size = none" to disable this unwanted limitation which
defaulted to creating zram-based swap with a maximum size of 4096MiB.

Fixes: dc8fa2bd ("common: Replace deprecated systemd-swap[1] with zram-generator")

Redirect fail2ban log to SYSLOG

Closes #322

See merge request !456

The upstream branch is set by the earlier "git pull --set-upstream".

Ref #374