We had a GeoIP mirror in the past based on nginx and its GeoIP module,
but it didn't perform very well, due to the high latency (asking a
central server for the package and then redirected to the closest
mirror).

One of the reasons for offering this service, is so we can relieve
mirror.pkgbuild.com which is burning a ton of traffic (50TB/month),
likely due to it being the default mirror in our Docker image. Another
reason is so we can offer a link to our arch-boxes images in libosinfo
(used by gnome-boxes, virt-install and virt-manager), with good enough
performance for most users.

This time we take a different approach and use a DNS based solution,
which means the latency penalty is only paid once (the first DNS
request). The downside is that the mirrors must have a valid certificate
for the same domain name, which makes using third-party mirrors a
challenge. So for now, we are just using the sponsored mirorrs
controlled by the DevOps team.

Fix #101

With the PHP->Python port done[1][2], there isn't much need for aur-dev
anynmore. Most things can also be tested locally and aur-dev haven't got
any love since the port (ex: allowing the aurweb maintainers to deploy
without asking DevOps).

[1] https://lists.archlinux.org/pipermail/aur-general/2022-February/036786.html
[2] !525

Kind of sensitive information that doesn't need to be available to all
hosts.

Change docs/ssh-known_hosts.txt to be partially managed by Ansible, so
custom entries can be added to the top of the file. Use the new format
to write down the host keys of our two borg hosts.

Using GitLab's official backup tool takes too much time and, more
importantly, space; /srv/gitlab is a bit over 430G but backing it
up nearly exhausts its 1TB volume.

As we're creating btrfs snapshots and backing those up with borg, it
seems unnecessary to also create tarballs of the same data. GitLab's
documentation mentions snapshots as a viable backup strategy, and to
the restored system it should seem like recovering from a power loss.

[1] https://docs.gitlab.com/ee/raketasks/backup_restore#alternative-backup-strategies

Collects the smart data using smartctl and outputs them in the
textcollector dir. This expects smartd to be configured to regularly
self tests on a regular interval to detect if a disk is broken.

These are already known (so no need to hide them) and are fairly static
(so variables are more of a hindrance) so it's better to use the actual
usernames in the documentation. Also, simplify the first example given.

New username; separate and longer account manager + storage passwords.

Also, have to use --remote-path=borg1 when interacting with rsync.net.

prometheus-borg-textcollector is no longer started by timer, but instead
defines a WantedBy= relationship with the borg-backup{,-offsite} service.

Jelle van der Waa authored 4 years ago and Sven-Hendrik Haase committed 3 years ago

Document how we backup our databases/gitlab instances.

Disabled in:
0ae67c4a ("postfix: Disable STARTTLS Submission (port 587)")

CPU: Intel Xeon E5-2620 -> E-2288G
Disk: 2x~1TB -> 2x~500GB

The role for the clients is named postfix_null (per [1]) and it's much
simpler and cleaner than the postfix role. I hope can cleanup the
postfix role at a later date.

[1] http://www.postfix.org/STANDARD_CONFIGURATION_README.html#null_client

Add a default rate limit for 20 req/s for the uwsgi endpoint and
automatically ban users who reach this limit. The nginx-limit-req rule
does not ban users who reach the rss limit as these are not likely DoS
attempts.

The port was removed in:
4729ba40 ("postfix: Remove special "fast-path" smtpd")

https://lists.archlinux.org/pipermail/arch-dev-public/2021-July/030471.html

Fix #86

This offers improved separation between the server backups and should
avoid bumping against the storage box 10 concurrent connection limit.

Fixes: archlinux/infrastructure#362

nginx, certbot, postfix and mailman are still missing and the DNS is
still pointing to luna.

Kristian Klausen authored 3 years ago and Jelle van der Waa committed 3 years ago

[1] https://archlinux.org/news/move-of-official-irc-channels-to-liberachat/

Leonidas Spyropoulos authored 3 years ago and Jelle van der Waa committed 3 years ago

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>

Document how to whitelist some metrics for the public Grafana instance.

Closes: #334

Kristian Klausen authored 4 years ago and Jelle van der Waa committed 3 years ago

Fix #263

The file should not be on the main domain as it adds unnecessary
complexity to the archweb role and there is a bigger chance that we
unintentionally break connectivity checking (which has happened in the
past[1][2]).

This doesn't remove the file from the main domain[3], as we need to ship
a updated NetworkManager package first.

[1] https://www.reddit.com/r/archlinux/comments/keai0g/does_anyone_know_if_this_is_normal/
[2] https://www.reddit.com/r/gnome/comments/ke9ytm/network_manager_popup/
[3] http://www.archlinux.org/check_network_status.txt

Fix #239