GitLab

https://lists.archlinux.org/pipermail/arch-dev-public/2021-July/030471.html

Fix #86

Support creating support-staff accounts on mail.al.org for mail + create user for klausenbusk and denisse

See merge request !430

Primarily to be used for mail accounts on mail.archlinux.org.

Update the firewalld configuration as of 0.9.4.
MinimalMark/AutomaticHelpers options are deprecated and ignored. New
otions added.

Use sub-accounts for backups to Hetzner Storage Box

Closes #362

See merge request !434

This offers improved separation between the server backups and should
avoid bumping against the storage box 10 concurrent connection limit.

Fixes: #362

As it turns out, the total size does not include the snapshots (feature
which we recently enabled). Record the free space so we can have a more
accurate picture of the usable space remaining.

Now that we have enabled snapshots, summing all Borg repository sizes
has become a bad approximation of the total space used on the storage
box.

Migrate lists.al.org to a VPS

Closes #356

See merge request !424

Fix #356

The DNS is still pointing to luna.

nginx, certbot, postfix and mailman are still missing and the DNS is
still pointing to luna.

We want to use rspamd for lists.al.org at some point, so we can't
hardcode the domain to archlinux.org.

Use restrict key option and relative borg command

See merge request !433

No functional change; the "restrict" key option is a shorthand for:

- no-agent-forwarding
- no-port-forwarding
- no-X11-forwarding
- no-pty
- no-user-rc

It was added in OpenSSH 7.2 (2016-02-29) as a convenient way to specify
an authorized key should have "all current and future key restrictions"
applied to it.

Also switch to a relative borg command since its location is not really
standardized; on rsync.net it appears to be located under usr/local/bin
(though /usr/bin/borg works too, even if it doesn't exist!) and Hetzner
just forces its own command, ignoring ours. 🐱

The Borg documentation seems to agree with both the above alterations:

[1] https://borgbackup.readthedocs.io/en/stable/usage/serve.html

Create snapshot with current db and gitlab backups

See merge request !432

The helper scripts that create mysql/postgres database dumps as well as
the script running gitlab-backup were executed after the btrfs snapshot
was taken. This resulted in stale db and gitlab backups (from last run).

Move execution of these helper scripts further up so their outputs get
included in the btrfs snapshot.
Reported-by: Kristian Klausen <kristian@klausen.dk>

Currently only tracks repo sizes and total backup space used (as a sum
of the former, not the real exact space used on each storage location).

"In advanced mail filtering rule sets, it is useful to keep state or
configuration details across rules.  This document updates the Sieve
filtering language (RFC 5228) with an extension to support variables.
The extension changes the interpretation of strings, adds an action to
store data in variables, and supplies a new test so that the value of a
string can be examined."[1]

[1] https://datatracker.ietf.org/doc/html/rfc5229/

Fix #368

A new editor was introduced in the most recent mediawiki update which
comes with images from a new extension location

Fix deploying archweb on gemini

Monitor borg repo sizes

See merge request !431

gitlab-backup: Set GZIP_RSYNCABLE=yes so borg can dedup

See merge request !429

gitlab-backup produces gzipped tarballs that cannot be meaningfully
deduplicated by borg.  This can be mitigated by passing --rsyncable
to gzip.

The above is verified by creating two new borg repositories, adding
the two most recent gitlab.archlinux.org archives to both, with the
difference of re-compressing the tarballs with `gzip -1 --rsyncable`
before adding them to the second repository.

In the first case, the 215.97 GB backup archive gets compressed and
deduplicated down to 176.24 GB. With --rsyncable it gets reduced to
just 12.79 GB. These numbers are for /srv/gitlab/data/backups only,
but the other non-tarballed files get sufficiently deduped already.

Based on the above, I am hoping to see the borg repository for gitlab
shrink over time from the current 3 TB to around 600 GB which is more
manageable.

Loki is using a lot of storage (~8GB per week).

gitlab-pages uses the API source by default now[2] but DNS resolution is
broken[3] so we need to use the disk config source.
Reported upstream here[4].

[1] https://docs.gitlab.com/ee/administration/pages/#gitlab-pages-doesnt-work-after-upgrading-to-gitlab-140-or-above
[2] https://gitlab.com/gitlab-org/gitlab-pages/-/commit/9a30f40a785ffe586a79a75ebebabda5f513b76f
[3] https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/4243
[4] https://gitlab.com/gitlab-org/gitlab/-/issues/331699#note_608473496