#377

syncarchive: Drop --delay-updates; breaks incremental scan

See merge request !472

The --delay-updates option results in 6G memory usage per archive mirror
for a total of ~18G memory used on gemini when all three archive mirrors
are syncing. Less important (but still revelant!) is the memory usage on
each mirror, which climbs to about 11G during each synchronization.

Removing the --delay-updates option should be OK considering the archive
hosts data that almost never changes. Without this option, rsync is able
to do a sequential scan which uses 90M of memory (per archive mirror) on
gemini and about 250M on each mirror individually.

syncrepo tweaks now that rsync has fixed the stale .~tmp~ issue

See merge request archlinux/infrastructure!471

This reflects the recent changes made to syncrepo.

Using a temporary directory outside of /srv/ftp was meant to protect
against incomplete files from being synced by downstream mirrors. It
does not achieve this to much effect though; each file gets uploaded
to the temporary directory but then immediately moved under a .~tmp~
directory at its target location (.~tmp~ because of --delay-updates,
otherwise the file would be renamed to its final path).

The `--delay-updates` option by itself sufficiently protects against
temp files being transferred to downstream mirrors; when used by the
receiver, it automatically adds an exclude rule for ~.tmp~, behaving
exactly like we want it to. As such, the `--temp-dir` option doesn't
provide any further benefit and can be removed.

The workaround can be skipped/removed when using rsync newer than 3.2.3.

- Replace --delete-after with more efficient --delete-delay.
- Move "-p" together with the other short options.
- Remove reference to empty ${VERBOSE} variable.

This reverts commit 75f9ca3c.

This should be fixed in rsync versions newer than 3.2.3. In Arch the fix
has been shipped in the rsync 3.2.3-4 package, which our own mirrors now
have been updated to.

[1] https://github.com/WayneD/rsync/issues/192

The DNS resolution issue has been fixed[1][2].

[1] https://gitlab.com/gitlab-org/gitlab/-/issues/331699#note_635123263
[2] https://gitlab.com/gitlab-org/gitlab-pages/-/merge_requests/513

Fix #380

This reverts commit a863917f.

Remove syslog-ng related configs

See merge request !469

Benedict Baskar authored 3 years ago and Kristian Klausen committed 3 years ago

Fix #376

Add robots.txt for archmanweb

Closes archmanweb#32

See merge request !470

Ref #381

Reverts caching issue

Archiving arch-commits mails maxes out the single vCPU of CX11 and
results in High CPU Prometheus alert. If we decide not to maintain
mail archive for arch-commits, then we can likely scale back down.

Replace runner1 with a new bigger box

See merge request !463

CPU: Intel Xeon E5-2620 -> E-2288G
Disk: 2x~1TB -> 2x~500GB

Split storage box monitoring into new text collector

See merge request !466

This was previously monitored as part of the borg text collector, but
now that it only runs after each backup (instead of hourly) the stats
from monitoring.archlinux.org do not remain accurate for long. Switch
back to hourly checks of the storage box's disk usage by adding a new
text collector just for this purpose.

Run borg-textcollector after each backup completes

See merge request !465

Instead of gathering borg statistics every hour or so, run the text
collector script only once after each borg-backup service finishes.

Also split the borg text collector script into two similar scripts,
where each one gathers borg statistics for its respective borg host.

Use RandomizedDelaySec=30min in Borg TextCollector

See merge request !464

Doing this in an attempt to be kind to our Borg hosts in cases where the
prometheus-borg-textcollector.timer is restarted on all hosts and avoids
having all machines querying the Borg hosts within the same minute. Only
downside is that the timers will trigger every 75-ish minutes instead of
exactly every hour, but this should not be a problem.

Split the postfix role into a role for mail.a.o and the clients

See merge request !454

The role for the clients is named postfix_null (per [1]) and it's much
simpler and cleaner than the postfix role. I hope can cleanup the
postfix role at a later date.

[1] http://www.postfix.org/STANDARD_CONFIGURATION_README.html#null_client

Fixes: cf9c92fd ("dovecot: Disable POP3")

Implicit TLS is the future[1].

[1] https://datatracker.ietf.org/doc/html/rfc8314

No one uses it and less to worry about.

Fix #205

Restrict the mail users to passwd and decouple the mailboxes from the system user

See merge request !450

The homedir is now /home/vmail/%d/%n instead of /home/$USER.

Preparation for switching to a virtual user setup and removing all the
staff users from mail.a.o.

The users are only meant as a way to change the mail password and
setting up forwarding (~/.forward), the latter will be handled by the
DevOps team now.

Fixes: 678845af ("Add Kape server IPv6 addresses (fixes #230)")