Jakub Klinkovský authored 4 years ago and Sven-Hendrik Haase committed 4 years ago

The original command does not work due to an import error.

Add prometheus-memcached-exporter

See merge request !90

Extend the memcached service for the AUR to allow the memcached group to
read the socket to obtain statistics.

Document Prometheus monitoring

See merge request !89

Remove configuration and entries for retired PIA boxes

See merge request !87

Jelle van der Waa authored 4 years ago and Sven-Hendrik Haase committed 4 years ago

The PIA boxes are retired.

Warn 25 days before certificate expires instead of 3 days before

See merge request !84

Kristian Klausen authored 4 years ago and Sven-Hendrik Haase committed 4 years ago

3 days is a bit too late. Certbot renews the certificate 30 days before,
so 25 days should be safe and shouldn't cause any "false positives" due
to transient errors.

/srv/http/archweb has to be readable for nginx to serve css/js static
assets.

Ensure the Keycloak custom theme background works in all login related pages

Closes #136

See merge request !83

Closes: #131

kernel: further default sysctl hardening

See merge request !81

- unprivileged bpf: we do not need this on our infra, we can assume
  bpf() calls will happen with CAP_SYS_ADMIN if required.

- unprivileged userns: we do not need this on our infra for none of
  our services or similar. Reduce attack surface by a huge margin
  including most recent CVE-2020-14386.

- kptr restrict: we already check for CAP_SYSLOG and real ids but we
  really do not require any specific kernel pointers to be logged.
  Settings this to 2 instead to blank out all kernel pointers to
  protect against info leak.

- kexec: disable kexec as we do never want to kexec our running servers
  into something else. Settings this sysctl disables kexec even if its
  compiled into the kernel.

- bpf jit harden: harden BPF JIT compiler to mitigate JIT spraying for
  the sacrifices off a bit performance for all users including
  privileged.

The prometheus-mysqld-exporter connects over localhost to collect
stats, so networking has to be enabled. mariadb's default is to serve on
0.0.0.0, so change the configuration to serve on localhost.

Improve mariadb configuration

See merge request !79

Jakub Klinkovský authored 4 years ago and Jelle van der Waa committed 4 years ago

The default value is 128M and our servers have plenty of RAM for that.

Jakub Klinkovský authored 4 years ago and Jelle van der Waa committed 4 years ago

The upstream default value is 2000 since 10.1.7:
https://mariadb.com/kb/en/server-system-variables/#table_open_cache

See also commit f164d000

We switched for monitoring to prometheus so zabbix-agent is unwanted and
we don't want to accidently deploy it again.

Add Support Staff subgroups in Keycloak

See merge request !57

Expand the Support group with subgroups for the Wiki, Forum, Security
Tracker and Archweb. The subgroups are just a placeholder for groups for
the roles which a user can be in for the service. New onboarded users
should be assigned to correct groups for their Support staff team.

Grafana

See merge request !73

Jelle van der Waa authored 4 years ago and Sven-Hendrik Haase committed 4 years ago

Configure Grafana to use Keycloak OpenID Connect for authentication. For
now only DevOps is configured as admin and Arch Staff as general Viewer
roles.

Jelle van der Waa authored 4 years ago and Sven-Hendrik Haase committed 4 years ago

To show the session IP address in /profile in Grafana the
X-Forwarded-For header has to be set.

Jelle van der Waa authored 4 years ago and Sven-Hendrik Haase committed 4 years ago

As we are moving to prometheus it's no longer required.

Use IPs from Hcloud

See merge request archlinux/infrastructure!82

Now that we manage DNS via Terraform and Hetzner DNS API, it makes sense to use the data provider from
hcloud to get the server IPs.