Add a new role called prometheus_exporters which should be run on every
machine we have and starts different collectors depending on what group
the machine is in. Currently supported our the gitlab runner exporter,
rebuilder textcollector, mysqld-exporter, borg textcollector and an
node/arch exporter. The arch exporter monitors the security status and
pacman out of date packages gauge.

aurweb: serve static assets with nginx and use Cache-Control

See merge request archlinux/infrastructure!78

archwiki: fix directory permissions

See merge request archlinux/infrastructure!64

- home directory needs 751 - nginx accesses it to serve static files
- cache and sessions directories are used only by PHP -> 750
- uploads is public -> 755

Note that the "fix home permissions" task was duplicated. Other tasks
fixing permissions were moved above.

Small changes for testing some roles in local containers

See merge request archlinux/infrastructure!58

This is much cleaner because the nginx role does not have to set the
fastcgi_cache variable to "false" by default, which was overridden by
host_vars/apollo.archlinux.org to "wiki", but the value was still
hardcoded in the config.

At first, I was wondering that the cache "zone" name should be
generalized to improve the configuration (from the original per-host to
per-service), but that would be an overkill since the fastcgi cache is
used only for the wiki...

This is needed for the role to work in containers. The option will also
be applied upstream, see the upstream ticket:
https://github.com/smartmontools/smartmontools/issues/62

The previous task creating the "zabbix_agent" user in the database stays
here as it actually needs the mariadb role. But note that it uses a
hardcoded name "zabbix_agent" for setting the password. The zabbix_agent
uses a different variable ("{{zabbix_agent_mysql_password}}") in the
my.cnf.j2 template, but I don't see where the variable is defined...

This role actually uses a handler from nginx to reload nginx.service.

Otherwise the timer may be started before mysqld and the service would
fail at the first start.

archwiki-runjobs.service is one-shot and timer-activated, it is not
supposed to be enabled.

This happens in the local Docker container, not sure about the
production environment...

Increase AUR machine size

See merge request archlinux/infrastructure!76

Exclude btrfs docker submount from being backed up

See merge request archlinux/infrastructure!75

The backup script fails on our Gitlab machine as it tries to remove a
snapshot of the docker submount, which we do not want to backup anyway.

btrfs subvolume snapshot -r /var/lib/docker/btrfs /var/lib/docker/btrfs/backup-snapshot
ERROR: Not a Btrfs subvolume: Invalid argument

It's per worker, and we have 5 of them.

Add prometheus role for the prometheus/alertmanager server

See merge request archlinux/infrastructure!71

Introduce a new monitoring server with prometheus and alertmanager for
monitoring all our boxes.

Add PTR entries for IPv6

See merge request archlinux/infrastructure!70

Update seblu public ssh key

See merge request archlinux/infrastructure!69

Make custom Keycloak theme responsive

See merge request archlinux/infrastructure!67

Remove vesath & retire devops

See merge request archlinux/infrastructure!68