Benedict Baskar authored 3 years ago and Kristian Klausen committed 3 years ago

Fix #376

Add robots.txt for archmanweb

Closes archmanweb#32

See merge request archlinux/infrastructure!470

Ref #381

Reverts caching issue

Archiving arch-commits mails maxes out the single vCPU of CX11 and
results in High CPU Prometheus alert. If we decide not to maintain
mail archive for arch-commits, then we can likely scale back down.

Replace runner1 with a new bigger box

See merge request archlinux/infrastructure!463

CPU: Intel Xeon E5-2620 -> E-2288G
Disk: 2x~1TB -> 2x~500GB

Split storage box monitoring into new text collector

See merge request archlinux/infrastructure!466

This was previously monitored as part of the borg text collector, but
now that it only runs after each backup (instead of hourly) the stats
from monitoring.archlinux.org do not remain accurate for long. Switch
back to hourly checks of the storage box's disk usage by adding a new
text collector just for this purpose.

Run borg-textcollector after each backup completes

See merge request archlinux/infrastructure!465

Instead of gathering borg statistics every hour or so, run the text
collector script only once after each borg-backup service finishes.

Also split the borg text collector script into two similar scripts,
where each one gathers borg statistics for its respective borg host.

Use RandomizedDelaySec=30min in Borg TextCollector

See merge request archlinux/infrastructure!464

Doing this in an attempt to be kind to our Borg hosts in cases where the
prometheus-borg-textcollector.timer is restarted on all hosts and avoids
having all machines querying the Borg hosts within the same minute. Only
downside is that the timers will trigger every 75-ish minutes instead of
exactly every hour, but this should not be a problem.

Split the postfix role into a role for mail.a.o and the clients

See merge request archlinux/infrastructure!454

The role for the clients is named postfix_null (per [1]) and it's much
simpler and cleaner than the postfix role. I hope can cleanup the
postfix role at a later date.

[1] http://www.postfix.org/STANDARD_CONFIGURATION_README.html#null_client

Fixes: cf9c92fd ("dovecot: Disable POP3")

Implicit TLS is the future[1].

[1] https://datatracker.ietf.org/doc/html/rfc8314

No one uses it and less to worry about.

Fix #205

Restrict the mail users to passwd and decouple the mailboxes from the system user

See merge request archlinux/infrastructure!450

The homedir is now /home/vmail/%d/%n instead of /home/$USER.

Preparation for switching to a virtual user setup and removing all the
staff users from mail.a.o.

The users are only meant as a way to change the mail password and
setting up forwarding (~/.forward), the latter will be handled by the
DevOps team now.

Fixes: 678845af ("Add Kape server IPv6 addresses (fixes #230)")

Add redirects for git.archlinux.org using a map

See merge request archlinux/infrastructure!439

Increase zram-fraction to 1.0 for lists.archlinux.org

See merge request archlinux/infrastructure!460

It's been running out of swap during borg-backup and seems to get good
compression ratios; try upping the zram size to 100% of RAM (from 50%).

Add fail2ban exporter

See merge request archlinux/infrastructure!457

The fail2ban exporter exports the amount of bans per jail.

hcloud_inventory: Optimize --list to avoid --host calls

See merge request archlinux/infrastructure!459

By adding a top-level element called "_meta" to the --list response,
Ansible will not call the inventory script with --host for each host
thus saving a lot of time and many requests to the Hetzner Cloud API.

The speed-up is significant; `ansible-inventory --list` is down from
about 1 minute to just 7 seconds in my testing (with ~60ms latency).

As per the following deprecation warning (even though it has a typo):

[DEPRECATION WARNING]: [defaults]callback_whitelist option, normalizing
names to new standard, use callback_enabled instead. This feature will
be removed from ansible-core in version 2.15.  Deprecation warnings can
be disabled by setting deprecation_warnings=False in ansible.cfg.

[1] https://github.com/ansible/ansible/pull/74845