Merge branch 'new-uk-geo-mirror' into 'master'

geomirror: new uk based mirror sponsored from jump.net.uk

See merge request archlinux/infrastructure!731

docs/geomirrors.md

+# Geo mirrors
+
+DevOps team maintain a geo mirror across the world. The Geo mirror is public facing on geo.mirror.pkgbuild.com domain and it will resolve the closest to the location of the requester mirror.
+
+## Locations
+
+| Mirror | Location |
+| ----------- | ----------- |
+| https://sydney.mirror.pkgbuild.com/ | Australia |
+| https://europe.mirror.pkgbuild.com/ | Czechia |
+| https://asia.mirror.pkgbuild.com/ | Hong Kong |
+| https://seoul.mirror.pkgbuild.com/ | South Korea |
+| https://london.mirror.pkgbuild.com/ | United Kingdom |
+| https://america.mirror.pkgbuild.com/ | United States |
+
+### Logical split
+The continent mirrors america, asia and europe contain the archive mirrors as well as repository mirrors. The city mirrors have just the repositories hosted.
+
+## Requirements
+- Host with Arch Linux installed
+- root access provided
+- Enough storage to host repos / debugrepos (at least)
+- Bandwidth (depends on location)   
+
+## Adding a new mirror box
+- Add new entries in `hosts` file under `mirrors` and `geo_mirrors` sections
+- Adjust terraform `tf-stage1/archlinux.tf` to include the IPv4 and IPv6 entries of the new server
+- Adjust terraform `tf-stage1/templates.tf` to include the IPv4 and IPv6 entries of the new server as a `NS` record for `geo.mirror.pkgbuild.com`
+- Add a new files in `host_vars`
+    - `host_vars/<fqdn>/misc`
+        Containing all the information for the mirror itself
+    - `host_vars/<fqdn>/vault_wireguard.yml`
+        Containing the wireguard private key in encrypted vault
+
+## Ansible Playbooks execution
+
+| Playbook | Roles | Reason | Hosts (limits) |Comments |
+| ----------- | ----------- | ----------- | ----------- |  ----------- |
+| install_arch | All | Install Arch | | Optional if you can |
+| mirrors.yml | All | Setup mirror | `<fqdn>` | |
+| redirect.archlinux.org.yml | acme_dns_challenge | Make TXT records | | |
+| gemini.archlinux.org.yml | dbscripts | Allow debug repo syncing | | |
+| mirrors.yml | geo_dns | Add new domain to DNS | All other mirrors from geo.mirror | |
+| monitoring.archlinux.org.yml | wireguard,prometheus | Allow loki and prometheus to fetch data | | |
+| archlinux.org.yml | postgres,wireguard | Allow wireguard IP to connect for Mirror check | | Optional see Check Location below |
+
+### Add mirror in geo.mirror.pkgbuild.com
+
+Add mirror IP and FQDN in archweb admin https://archlinux.org/admin/mirrors/mirror/ under the `geo.mirror.pkgbuild.com` entry.
+
+### Check Location (optional)
+
+If you want the server to check for ping and stats create an entry in:
+ https://archlinux.org/admin/mirrors/checklocation/

docs/wireguard.md

+# WireGuard
+
+Many of our servers communicate through wireguard VPN with each others. If you need to collect logs with `loki` and metrics with `prometheus` for dashboards you need to have a wiregauard IP.
+
+## Setting up
+1. For a new server add a new unused wireguard IP and set the following in `host_vars/<fqdn>/misc`
+    ```
+    wireguard_address: <wg-ip>
+    wireguard_public_key: <wg-pubkey>
+    ```
+
+1. Save the private key in a encypted vault in  `host_vars/<fqdn>/vault_wireguard.yml`
+
+    Tips: 
+    - Pick next available IP for Wireguard from `grep -r wireguard_address host_vars/ | cut -f3 -d: | sort -h`
+
+    - Wireguard key generation docs: https://www.wireguard.com/quickstart/#key-generation 
+1. Execute `wireguard` and `prometheus` roles on `monitoring.archlinux.org.yml` playbook to get data from the server

host_vars/london.mirror.pkgbuild.com/misc

+hostname: "london.mirror.pkgbuild.com"
+mirror_domain: "london.mirror.pkgbuild.com"
+archweb_mirrorcheck_locations: [22, 23]
+ipv4_address: "185.73.44.89"
+ipv4_netmask: "/22"
+ipv4_gateway: "185.73.44.1"
+ipv6_address: "2001:ba8:0:4030::2"
+ipv6_netmask: "/64"
+ipv6_gateway: "fe80::fcff:ffff:feff:ffff"
+filesystem: "btrfs"
+network_interface: "enX0"
+system_disks:
+  - /dev/xvda1
+extra_disks:
+  - /dev/xvdb
+wireguard_address: 10.0.0.43
+wireguard_public_key: FuhMj8Vrk0HUR10O2dmgeXtw+bMAuhNesYD+h0lKgSc=

host_vars/london.mirror.pkgbuild.com/vault_wireguard.yml

+$ANSIBLE_VAULT;1.1;AES256
+30663137336130633937383231343062333664636631323739373164663563363565383435633362
+6331356532616630373432373031383139613633646461330a373936663337326633656464353862
+36326239373864383662343737313436653639383562303433363634323562653637373236653736
+3765663532643338650a626433353131353730623864646535646138333236316563353032616235
+38653765306433656539383533653930376564663361356134303539316335636435616130383234
+63346238323761343635326263396362656663363237336232663039346465656265616366373433
+36353862386661366563366535383439333531656564366238323032656232633462336166343766
+37613432323131623461

hosts

@@ -14,6 +14,7 @@ sydney.mirror.pkgbuild.com
 america.mirror.pkgbuild.com
 asia.mirror.pkgbuild.com
 europe.mirror.pkgbuild.com
+london.mirror.pkgbuild.com
 mirror.pkgbuild.com
 seoul.mirror.pkgbuild.com
 sydney.mirror.pkgbuild.com
@@ -22,6 +23,7 @@ sydney.mirror.pkgbuild.com
 america.mirror.pkgbuild.com
 asia.mirror.pkgbuild.com
 europe.mirror.pkgbuild.com
+london.mirror.pkgbuild.com
 seoul.mirror.pkgbuild.com
 sydney.mirror.pkgbuild.com
 

tf-stage1/archlinux.tf

@@ -353,6 +353,10 @@ locals {
       ipv4_address = "89.187.191.12"
       ipv6_address = "2a02:6ea0:c237::2"
     }
+    "london.mirror" = {
+      ipv4_address = "185.73.44.89"
+      ipv6_address = "2001:ba8:0:4030::2"
+    }
     "seoul.mirror" = {
       ipv4_address = "145.40.87.75"
       ipv6_address = "2604:1380:11:2600::1"

tf-stage1/templates.tf

@@ -270,3 +270,13 @@ resource "hetznerdns_record" "geo_ns5" {
   type    = "NS"
   ttl     = lookup(local.geo_domains[each.key], "ttl", 86400)
 }
+
+resource "hetznerdns_record" "geo_ns6" {
+  for_each = local.geo_domains
+
+  zone_id = lookup(each.value, "zone", hetznerdns_zone.archlinux.id)
+  name    = each.value.name
+  value   = "london.mirror.pkgbuild.com."
+  type    = "NS"
+  ttl     = lookup(local.geo_domains[each.key], "ttl", 86400)
+}