Playbook allows us to provision an aurweb sandbox host.

Ref: aurweb/!752

Signed-off-by: moson <moson@archlinux.org>

Introduce "root_additional_keys" variable allowing us to deploy
additional root keys with our "root_ssh" role

Signed-off-by: moson <moson@archlinux.org>

Use variables to define our systemd unit files.

Signed-off-by: moson <moson@archlinux.org>

* Move modules installation:
We need some modules to be installed when doing the DB init. (alembic)
* Remove double entry for starting "aurweb-git-archive.timer"
* Link update wrapper after creating git repo
* Fix permissions cgit deploy

Signed-off-by: moson <moson@archlinux.org>

dbscripts: fix createlinks for filenames that contain spaces

Closes #524

See merge request !751

So far the for loop recognized filenames with spaces as different words:

$ for f in $(find pkg -type f); do echo "$f"; done
pkg/usr/bin/Surge
XT
Effects
pkg/usr/bin/Surge
XT

While the correct output here would have been:
pkg/usr/bin/Surge XT Effects
pkg/usr/bin/Surge XT

We fix this by just passing everything directly to readelf, which also
removes the loop overhead. This results in a significant speedup for
packages with a lot of libraries and binaries.

fixes: #524


Co-Authored-By: Evangelos Foutras <evangelos@foutras.com>

This allows for tasks/include/upgrade-server.yml to be reused elsewhere.

Commit 8e6d5474 ("sshd: use drop-in for basic sshd configuration")
changed the sshd_config.j2 template to contain only overridden bits of
sshd_config. However, it did not account for the install_arch role use
of the same template which was still installed to /etc/ssh/sshd_config.

Fix install_arch to install to etc/ssh/sshd_config.d/override.conf too.

Fixes: 8e6d5474 ("sshd: use drop-in for basic sshd configuration")

Misc changes for supporting aurweb's review apps need

See merge request !748

The VMs created in the Hetzner cloud sandbox project must be accessible
with a DNS name. This creates a dedicated DNS zone for this purpose.

For now this zone will only be used by the aurweb project, so it can
create DNS records for the VMs it creates. This is needed so the dynamic
created environment can be accessed over HTTPS.

[1] https://docs.gitlab.com/ee/ci/review_apps/

Aurweb wants to use terraform to create VMs in the Hetzner cloud sandbox
project and it must store the terraform state somewhere. The state can
be stored in GitLab[1], but unfortunately the access is not very
granular. So to avoid handing the CI pipeline too much access to the
aurweb project, a new project has been created, to store only the
terraform state, and an associated project access token.

[1] https://docs.gitlab.com/ee/user/infrastructure/iac/terraform_state.html

This is meant to be used in the Hetzner cloud sandbox project, so SSH
keys can be injected when a new VM is created from e.g. a CI pipeline,
so that the CI pipeline can SSH to the newly created VM.

The EC2 metadata service is used over the Hetzner metadata service, as
it is supported by more providers (including Hetzner).

A new Hetzner cloud project has been created called "Sandbox". This
project is meant for non-production workload which must be created
on-demand from e.g. a CI pipeline. The first project using the sandbox
is aurweb, which wants to use GitLab's Review apps[1] feature to create
dynamic environments on-demand.

Two API tokens have been created, one for the infrastructure project (to
be used by packer) and for the aurweb project.

[1] https://docs.gitlab.com/ee/ci/review_apps/

As of version 1.7.0, HCL2 is the preferred way to write Packer
templates. The documentation reflect this and it is easier if we use the
preferred format.

acme_dns_challenge: turn into more generic dyn_dns

See merge request !754

Extend the role (previously used for ACME DNS verifications only) to
support dynamic DNS functionality planned for sandbox.archlinux.page.

Setup bugbuddy server for upcoming bugbuddy tool

See merge request !743

Bugbuddy is the upcoming tool for assigning package bugs to the proper
folks. The bugbuddy role will be created at a later date when the tool
is ready.

grafana: Add requests to aurweb dashboard

See merge request !753

Add timeseries visualizations for number of requests by status and type.

Convert "graph" vis. types to "timeseries" for Users and Packages.
("graph" is deprecated)

Signed-off-by: moson <moson@archlinux.org>

This saves us from having to rebase on every upstream config change.

The same drop-in functionality is now provided by the openssh package
via /etc/ssh/sshd_config.d/.

Add RedHat account and dedicated GitHub account for archlinux-docker

See merge request !704

This is needed as archlinux-docker wants to push its container images to
GitHub Packages[1]. Unfortunately, the existing GitHub account has too
much access and it is not possible to limit the token to a single
repository[2].

[1] archlinux-docker#73
[2] https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry#authenticating-with-a-personal-access-token-classic

This is needed as archlinux-docker wants to push its container images to
Quay.io[1], which requires a RedHat account.

[1] archlinux-docker#73

Requested by @torxed to be able to commit from a pipeline.

[1] https://gitlab.archlinux.org/archlinux/teams/mirror-administrator/arch-mirrors

We are not sure where this CNAME was originally used. DNS records were
transferred to Terraform sometime in 2020 so we lack history for them.

aurweb: Dashboard changes & removal of textcollectors

See merge request !744

Signed-off-by: Leonidas Spyropoulos <artafinde@archlinux.org>

Signed-off-by: Leonidas Spyropoulos <artafinde@archlinux.org>

With aurweb!743 we expose those gauges directly from /metrics

Signed-off-by: moson <moson@archlinux.org>

Rename "search_requests_total" to "aur_search_requests_total"

Related MR: aurweb!743

Signed-off-by: moson <moson@archlinux.org>

We have 100 max connections to Libera.

archbuild: Add systemd conf for devtools resource control

See merge request !745

Leveraging devtools 1.0.4, install systemd snippets which
  - distribute CPU and IO equally between users,
  - distribute CPU and IO equally between build containers of a user,
  - prefer distributing CPU and IO to things that are not build
    containers, and
  - copy our user oomd config for users to apply to build containers.