Add Kape donated servers

Setup Kape servers as archive mirrors (asia,europe,america), Gitlab
runner and Rebuilderd worker. All machines except runner1 are EFI
machines with grub setup and a EFI parition which is not supported by
our ansible install role and is manually rolled out.

docs/kape.md

+## Kape Servers
+
+All donated servers are with EFI except runner1.archlinux.org.
+
+### Archive mirrors
+
+Three servers have set up as archive mirrors:
+
+* america.mirror.pkgbuild.com
+* asia.mirror.pkgbuild.com
+* europe.mirror.pkgbuild.com
+
+The servers have been setup as RAID 5 with 3 x 10TB disks.
+
+### Gitlab runner
+
+A runner is setup on a 2xE5-2620v4 -  64GB - 2x 1TB SSD as runner1.archlinux.org.
+
+### Rebuilderd worker
+
+A rebuilderd worker is setup on a EPYC - 256GB - 2x 500GB SSD as repro1.pkgbuild.com

docs/rebuilderd.md

@@ -6,7 +6,8 @@ Results are shown on [our website](https://reproducible.archlinux.org) which is
 
 ## Configuration
 
-Setting up rebuilderd-workers requires adding the `rebuilderd_worker` role to the playbook and adding `rebuilderd_workers` a list with rebuilderd-worker names for example:
+Setting up rebuilderd-workers requires adding the new machine under `rebuilderd_workers` in hosts
+and adding `rebuilderd_workers` with a list of rebuilderd-worker names for example too it's host_vars:
 
 ```
 rebuilderd_workers:
@@ -14,6 +15,8 @@ rebuilderd_workers:
  - repro12
 ```
 
+Then run the rebuilderd-workers playbook.
+
 ## Monitoring
 
 The rebuilderd workers and queue are monitored by Prometheus.

docs/ssh-hostkeys.txt

@@ -9,6 +9,17 @@
 256 MD5:e7:c7:62:0a:d1:b7:24:62:08:15:73:18:9c:9e:5e:a7 root@archlinux-packer (ED25519)
 3072 MD5:9b:04:00:69:a4:f2:62:c3:80:8e:a8:b1:80:ad:a7:9d root@archlinux-packer (RSA)
 
+# america.mirror.pkgbuild.com
+1024 SHA256:pycjsXlenFbGqHMp2C5tJZRKJnxCZ1usCux9NOJVTQA root@america.mirror.pkgbuild.com (DSA)
+256 SHA256:cmT+nhDEvcuWeZhc5q8GVv6xuxmtS5PkL1ehsilU0C0 root@america.mirror.pkgbuild.com (ECDSA)
+256 SHA256:046/o/xSGSruNAEhIMaW2E4a56i7l2jZe13nJADwczo root@america.mirror.pkgbuild.com (ED25519)
+3072 SHA256:c/0AQtR2RlNTNI4fHdki6ef7/fWekT87sJ1B8ODHc/Q root@america.mirror.pkgbuild.com (RSA)
+
+1024 MD5:24:c0:a6:27:87:f5:04:c5:e5:89:58:1c:e8:a9:06:9d root@america.mirror.pkgbuild.com (DSA)
+256 MD5:86:d5:e0:ed:d7:3e:56:50:0a:92:60:21:53:24:4d:0f root@america.mirror.pkgbuild.com (ECDSA)
+256 MD5:4b:0b:1c:81:27:81:7a:22:b4:48:88:75:69:a5:b4:4e root@america.mirror.pkgbuild.com (ED25519)
+3072 MD5:a2:41:dc:97:5a:ae:89:7a:4f:69:f7:ec:a0:d4:67:b6 root@america.mirror.pkgbuild.com (RSA)
+
 # apollo.archlinux.org
 1024 SHA256:WArxFzvhf5HknYxil2EQSHHRirM2cyjqbtLvhbQAYC8 root@apollo (DSA)
 256 SHA256:sYJfY17PE0kJ4K8fbkPK/XqRQjY1+g6hmIF7dvTbZoo root@apollo (ECDSA)
@@ -20,6 +31,17 @@
 256 MD5:a7:84:8b:95:4f:53:ac:b6:9d:24:79:79:fc:c7:bf:1f root@apollo (ED25519)
 2048 MD5:77:b0:17:18:57:74:38:91:47:31:43:04:47:e9:9e:30 root@apollo (RSA)
 
+# asia.mirror.pkgbuild.com
+1024 SHA256:NZilDXhhVEFsT7JPcB6APY8HhiO7RgyRMyX3pL+zDik root@archive1.mirror.pkgbuild.com (DSA)
+256 SHA256:gMJUYOIH8zdYa1x92WnrlLkxZtTf99Na+ESnZ+Kvk2E root@archive1.mirror.pkgbuild.com (ECDSA)
+256 SHA256:aKSZxnj43Q0c3CZ82KOBzV6/I6xH1K0SEg2l3nTpbB4 root@archive1.mirror.pkgbuild.com (ED25519)
+3072 SHA256:xJG12dFONxe7TNST9oogoO4nEWprHV2o/92FbPT4E6I root@archive1.mirror.pkgbuild.com (RSA)
+
+1024 MD5:16:e8:82:51:1f:cd:5d:bf:08:13:68:40:37:bc:e0:fa root@archive1.mirror.pkgbuild.com (DSA)
+256 MD5:c0:3a:eb:cb:b7:47:52:01:e3:cb:ab:40:94:b3:a4:21 root@archive1.mirror.pkgbuild.com (ECDSA)
+256 MD5:f9:3b:1f:ac:be:b6:15:67:07:02:30:48:eb:c0:30:eb root@archive1.mirror.pkgbuild.com (ED25519)
+3072 MD5:84:04:71:14:38:34:e0:c4:a3:fa:7c:3f:ee:e2:ed:59 root@archive1.mirror.pkgbuild.com (RSA)
+
 # aur-dev.archlinux.org
 1024 SHA256:VzUmG0B+Yb1mrcXVnJI0dMECOgi+7oIwW2PhPvOUhkw root@archlinux-packer (DSA)
 256 SHA256:KFBhxP4afMhI5dqHMJwIrbuJ9/EzlHsdXxJ5cwiTMbg root@archlinux-packer (ECDSA)
@@ -75,6 +97,17 @@
 256 MD5:54:23:82:0d:e6:da:6c:d7:09:f1:f0:0f:49:5a:64:04 root@dragon (ED25519)
 2048 MD5:e1:e6:4f:72:31:a3:9a:2c:af:e0:0f:53:43:27:6b:df root@dragon (RSA)
 
+# europe.mirror.pkgbuild.com
+1024 SHA256:Oq3eikchfo8Wt6AUzWAiU1mDR24rXudJR/zqKBFnrMo root@europe.mirror.pkgbuild.com (DSA)
+256 SHA256:3S0HuO72jHUUrPM8BjfcjsB0FNXkubxovc7Sm5jZBjc root@europe.mirror.pkgbuild.com (ECDSA)
+256 SHA256:aqnPnq4WG/3xNuKOJlsuCGgPiH0RWavcQi/n/HO9h6Y root@europe.mirror.pkgbuild.com (ED25519)
+3072 SHA256:cJGscbI/w0iINNBpU+Q6jLtSlF2Y3hLPs/By8CzX4tM root@europe.mirror.pkgbuild.com (RSA)
+
+1024 MD5:f3:da:87:c4:b2:bc:da:be:1c:ce:a3:73:3c:da:ff:f4 root@europe.mirror.pkgbuild.com (DSA)
+256 MD5:44:ef:66:dc:e2:68:86:69:ad:74:22:a4:92:c6:5b:e1 root@europe.mirror.pkgbuild.com (ECDSA)
+256 MD5:bd:af:e2:cb:6b:fe:b6:60:73:b0:ba:7b:db:af:21:b7 root@europe.mirror.pkgbuild.com (ED25519)
+3072 MD5:57:a2:59:db:c7:07:4f:ac:91:9a:f8:db:7f:16:a7:d4 root@europe.mirror.pkgbuild.com (RSA)
+
 # gemini.archlinux.org
 1024 SHA256:F1Corf6i2U72yub+CIzzGHLOMVKVnjALh1YHM8gBjxE root@gemini.archlinux.org (DSA)
 256 SHA256:If51DkTftUpDAFz65totgDfTd/ddu/2w/RBZIHtY74U root@gemini.archlinux.org (ECDSA)
@@ -218,6 +251,17 @@
 256 MD5:15:31:37:f4:f2:e5:93:e7:28:8f:e2:db:6e:6d:24:e2 root@repro3.pkgbuild.com (ED25519)
 3072 MD5:d7:f7:8e:0b:bf:8f:58:83:33:ba:c8:22:4b:3c:50:96 root@repro3.pkgbuild.com (RSA)
 
+# repro2.pkgbuild.com
+1024 SHA256:sppthtBQD60z8f0bDUnoMUesg55M7/ez4qGXVUUDtRQ root@repro2.pkgbuild.com (DSA)
+256 SHA256:enqq08K6vQV8CcISu1upR3Ooa63HD6Z+PtRzMVArnTk root@repro2.pkgbuild.com (ECDSA)
+256 SHA256:CA71k+BRGrEEcLLVKqtUBU55th2W12Emq/x++zGtoH0 root@repro2.pkgbuild.com (ED25519)
+3072 SHA256:zQy/zasnSYXF5h863hxxjKy7xqw2HifboYGBb59g9Vg root@repro2.pkgbuild.com (RSA)
+
+1024 MD5:2b:3b:9a:9f:b0:fc:d4:20:8b:21:67:bf:f7:a4:c8:e7 root@repro2.pkgbuild.com (DSA)
+256 MD5:95:c1:25:1a:b3:46:f3:d2:9f:19:21:02:9b:e7:5f:9e root@repro2.pkgbuild.com (ECDSA)
+256 MD5:21:76:73:3b:ac:30:6d:f5:a5:f6:52:2e:13:dc:b4:cb root@repro2.pkgbuild.com (ED25519)
+3072 MD5:12:70:8a:d4:ef:a9:43:6e:6b:53:46:71:f7:96:ec:fb root@repro2.pkgbuild.com (RSA)
+
 # reproducible.archlinux.org
 1024 SHA256:3HoA8rGGureKWKaIZst+Dc6f7yrf3Wfn5PO1HFMl35E root@archlinux-packer (DSA)
 256 SHA256:Yp3GHPq6GZZWUBv13XPiXciAbDFPH4XnEVHzR3/whUw root@archlinux-packer (ECDSA)
@@ -229,6 +273,17 @@
 256 MD5:1c:37:46:02:c8:ea:3a:dd:5e:41:66:05:5e:18:27:6a root@archlinux-packer (ED25519)
 3072 MD5:42:0a:57:89:2a:ae:e6:c5:c1:ae:6a:a2:bd:3d:5f:dc root@archlinux-packer (RSA)
 
+# runner1.archlinux.org
+1024 SHA256:TgWGBKCSo3rkdYuP4mJzGtH+Ucub8dmEJXnnzrhUSwM root@84.17.49.250 (DSA)
+256 SHA256:2OEO/Pv1OxCLaVg9r8wHzTR7V3YTWEw//g7LJgIBMGs root@84.17.49.250 (ECDSA)
+256 SHA256:nXpZ/R3IuOEAasoIvCCTkdMlyNjCnWVTvQBh6xvycIg root@84.17.49.250 (ED25519)
+3072 SHA256:Ii5JwUoEKXV37rdjnsts4k3ppbEX+GHYjBC9mygdwY4 root@84.17.49.250 (RSA)
+
+1024 MD5:31:f6:9c:50:d2:80:f1:fa:b5:50:c5:06:3a:9e:80:cc root@84.17.49.250 (DSA)
+256 MD5:ad:cb:d6:53:cb:42:8d:33:26:99:33:fd:c9:19:a0:ea root@84.17.49.250 (ECDSA)
+256 MD5:c1:7f:43:ad:dc:69:17:b4:33:df:bb:94:00:22:33:5a root@84.17.49.250 (ED25519)
+3072 MD5:da:9f:ec:c4:f3:c1:63:6e:ca:f4:9b:ac:32:23:d9:74 root@84.17.49.250 (RSA)
+
 # runner2.archlinux.org
 1024 SHA256:tK3XeUrt/ahlj1fdl5tQgeVXticllUIaewK2y86Jvzc root@runner2.archlinux.org (DSA)
 256 SHA256:iJGAgTQWSQk97dlNpkBMEW3peysCDnNFFZO5TyEguvs root@runner2.archlinux.org (ECDSA)

docs/ssh-known_hosts.txt

@@ -3,11 +3,21 @@ accounts.archlinux.org ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIb
 accounts.archlinux.org ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBu3+qlfqd8FwqodNzem7cCVcNA5RQpidYHkDRPdsZzq
 accounts.archlinux.org ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDG+EUMwsfIuK/Nyi9zfL6WQdXG6V8A7pJ8hV8RPuD4/hTbLlPsAgNi26TSegtb5f9nXhsIYav27F1KqzAIM0SZnt+BwKXx47LE0BR1VR9es47qaayHYYtDazfRfHdiw/4+e7lIJv00e0xbemquKEC8rmwSmQe/NH/uTH+ZTsAWo4x3m7+XsXNEWtZXLJBxvImFMrXJnQAakN0vyOpbXTXyxirsEdJnkGen4X37lEDc2te9QnyDB4QoSRCTVdxHNmxqo/2N1QRK9g12I8RMqXRRC4TdCWIsluu+NAcrwfG9nmToaIaiP3k5fzHcBYuZ/qTtDlO39deRtCt3epDiTxa+zgIvOBVPUULp620OwBSikrVt1gWjN04zMvgS+d87LZKZYoDdFOd7+P7slwmhXA0J0VX1xVuBA/dFLhMLArU8Xzwpu4m6lSa69cwC2arndQZV6qOmhKH+kxUsrOmesfdDLgtqMcUOrWVzQcXhzyoJ5OjCcGCcHYgKl8nYrTi2HLU=
 
+# america.mirror.pkgbuild.com
+america.mirror.pkgbuild.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG9cA9oUrmtWMqP8dEGZRJjRxW+iHe0DOhxSL16kF5Kv9mht3mqktPL3Okbdw/YrBXPX/YTLrgOPaTmstfkGITI=
+america.mirror.pkgbuild.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMofe+VPkI+MKGWYkonc5IsTwVmf2OcX8atVgnXkjbqL
+america.mirror.pkgbuild.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+7ePbae+aA+W6JZxhce1qvxnTG6VIEtShFD+/4TlMNFzVxh8DcmD+sw+NCKFshGddKhh3n4TWO68XAxWN4KQzfpgYHgzcoyNR3T1zle2oWfTL29KU3/X4OIvQc3aeqnyAhHvu7vXDnWfZgC1uGu0QecAqJcRxEPkz15OJkZjr/zZt01BUa2y7OUQbSFRckeLoSPQeSNFOQ0hlWCdN0laoHsYPznyievuJJxWMTa9Wkamn+oVx1S95sF1a6PSN8LztCxxXKRTikpau+SCLpPiqd1kV4WXgOPWGoRvQ0PNzOICwmKEeCdw0QfzoNm70FftLvy/bFq7qa1tSvau/SFwRRdnDRlr7dZXLqbXp+si+GD80TL4scUH6LRQJkxL0Z5X/FKOPGutf8BhIovbhoSkiTyY51bDQIXLTCqm1sE2GrEI1CvT5lOvG43C2e8U7bnLw2hQ++GRUct89mI8f/8hHdDllUkUgfsksL7ObMTvkbF6NoCogZjHsTRtwaP0DU/M=
+
 # apollo.archlinux.org
 apollo.archlinux.org ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMicA8QKPeY1hp29QcTe25eT7yd+zOx1sj6o0F+XA/POc2TRsiSidJogCaf4e3wpw4T2ccb7ixnvGmy7hCAcngA=
 apollo.archlinux.org ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGz+b+or4nKpcXJgDjwt3LdO0EPk9Zw1z1W9L8rcV8UX
 apollo.archlinux.org ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHUYq82CCrnZhey7Hclhe79+s7YUZv/So1HWjoSAs8qObpJX4Mn3bwcILOoD1LE6VdkQu+tZwFpl8A1DrmKgpO++SEoFft77jgigzDbwEuSuBbP8eOo1zyDX1q3Sipecf41s6psY3bxcVbINAkm/PDFxpM8tEU+8TqpCupa5fNLimiwBk7fyncxbah+ACaLlm+f02Ku9pBcPfFzlsEoZBrncAyhx3bm4qXH/uYVOtBjzi6KrZYyEbXX+0LxRhxuELkhYqbNpyFIDfPKYgXc1pRHgAkS2CxZO2p1Uy1zJFC8edM3ma+I0Wn9+alGMHC6jCOm2iFT9THLS2NPJq67Yan
 
+# asia.mirror.pkgbuild.com
+asia.mirror.pkgbuild.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEiKfM3AnTrtz/FljXR/8QLh5213MSDLgC3NzevnV6t2yrfY0XIQ5+ddwjhEVphC1bCn5jbuRetc7wP4DolNit4=
+asia.mirror.pkgbuild.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAL29ty9q8ojSshxgwTPm1thbBX2ioli2LONf7SJa959
+asia.mirror.pkgbuild.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1IhNA7ozz1BdmRGI5BJMS2dkCoxfMf3Jr5Q/CEPe4MOoO13+DSBW7EAm/1stwkCTKSnmPqpQcAW+CZU8bx72S/8OChAssgZq/5AP82xcv4zZtarQqa95EddcvsG1kVcIT94Yd+dIGFMgNjJMbUkOthmaENaDnZKgS7gjs4B0Pd3NKXu9McM1+tgusc9nKL+2yxndnyip/dBFDl4UwJWl0YixXpKHMcnnjtpLQ24ZUFsaXqZ+cRSYGUR5h9zvRGjzgyl6gzAc98kfoCrKK28NBwgh086SGiYTe1TlTQYcMBPXEuvk9xRTgB2PWRRzyiYnOM0u+QAd/qTj7SO85g4PpmGP6J7kmZ0WGXyW/c9fwrINy9KYt/h5ZRcpxB/6bDP1FmxTKkfvcHCYMraH/Zfzc9AdgfawC+5ul0PZAxIyni6E9AlKjZKmOvSpHKtoV7DYAvM/AajY43QoRCMsjay4vro116yFZdsWFnqt0iWdhDzKFM30ovbC1WFujIdLdWl8=
+
 # aur-dev.archlinux.org
 aur-dev.archlinux.org ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLbwdgWiWIEZmTmMJtEAyDjvTqTY2hIZTevy3gjYXzTLS65D8Bav1ci6e1DUINflKMbW/n8b8f/Ac0d4Lv7BRm4=
 aur-dev.archlinux.org ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPmLqDFITP0IBdJ11sc4QtNHtu/PUwjo5yvVfvcKxCru
@@ -33,6 +43,11 @@ dragon.archlinux.org ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbml
 dragon.archlinux.org ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGnUsMuS0hc7ATduUudV93QOanFoyyCfj+kKjeD8RTxF
 dragon.archlinux.org ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcXC5AWSgbG+ZP8vsaK4rpKugPrm0i7K4RADIYF2KX1aoaHvMBInwpY+qb0eRcIvWy2AEeOmpXrsQGrcUxMqsMjWnsuqWvlRdHxWJZ01TmL8Ui8xtyQF372yEjxbMtn1Yg120jdv1vyZTviWtL66ihZN3n4gIjY9lXJcE4hsGh+llIkiAqsrMUDt0lKaw/AeiiLcwbN9liD55U1SUf+gAmALu2Qp5IgUM1OLwGuISL8DtYkbRjeryGgB58/m50/Icg28IMqJ8+Dx8Pr2dJr3RLC6AMyiT/rw/5qTVw1KRiUWzeUdLcphTsMTC021XAXp1JUwnbWyemX8SdjZ+SJ2uL
 
+# europe.mirror.pkgbuild.com
+europe.mirror.pkgbuild.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBASkHNbJZvfME9OAFLZpxoVMt7JfKhN8/VpH6JPRD8eRXfXc2Wt0YOZQGzJsrUNoFchEUUGeNxs7vmj8nwtfqGI=
+europe.mirror.pkgbuild.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF6sVYSrTIVj+xwyC5uJdFVU+X50tAIDEndsnGta75C7
+europe.mirror.pkgbuild.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDssgto7Lqfm63rjGMd8R2lf398W3UHJfOx6GhYBUVWYYdkcvjvMLHutZZaiOcPi7RjBsJEIt1Mk8XzesxpcF5eB1vIwvbCvLzbTflNxrEGPbdW916IITZdKeVT/JpqzzgVdttXmULW6uZ93hILlyobeaSCfeWjyyxsCX04Y1vde/LkpkM5+8SbgwyhWE7idKKILg+3MHJG8xK+CEAUGOHT5+FaexJXMPlMY+O/owjU3vBF1omA4kPDh99/OtWhS2ngUFHWJlEb1+Jidfu6naAomFKZEBI6BWd2td/ngn1JcFzrqyjs0hbsnjpb5RhQ1w/z95mLhMhDH+HtCFgAwTOjOOQ+bDcc0wbVwSUmJx7ctzuFWqfTP+ts7xWC5TOmBMfH9TRMr1+A5CS01Nds1w2RW3S+CSy/CQr5/g1+IjyTuJ4fyvJjYhuk6ouLPl7jISX3EmAtDkB5CmBkQKt3OEMGJlPJs0WpArsNT6CqxECxzTbv+Wp1IXQfcJwspl9ESUs=
+
 # gemini.archlinux.org
 gemini.archlinux.org ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOck5++3KyQYcMl1eknD9opwVewCdrBy4xOQF9eC3gcdq3ZKNyM0hCIwZ8sZbzchtKWiHjjlPtoN3mesohtHUzE=
 gemini.archlinux.org ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIG3ghrZI2G6zc3RKzouAOdgfs6TIRpKjcUqO3pmgRFqk
@@ -98,11 +113,21 @@ repro1.pkgbuild.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlz
 repro1.pkgbuild.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKrJCFjvo5svIzF5d7agm1/QlqyYlegxSX6GebZp6E5c
 repro1.pkgbuild.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD3HSegYNMLbTKoQzei6mNyEPdSNHAS8oA/9XKWPjwJ/TFaP/ouesAxCjVJ3UxIVPNTC78bVVf3IsjginT4CmUUOfiFhZ5LEY9DnLzJHny4SHZR0JhhHJFHYqGk4J46S5GOD48kMDAm7pfZU6zTydsEEc5ym+aEihN48x4SCPikQaPPEZVV8OwfqE7MH0/PWNMiytPmfte73E4+7gXmfD5lHz2pgdhLkcaHzhkarpMHJVMhqwb6hqX2VBoI+2eJZSY8WU0qzPE3mNRXqhr87Rz4pcVkSm+g9euhmFChN1tTyszo3Qd6vdDYZd5eiSOzPtjzmIAYIrFVCZYjKW8aw7YhvujjDrl1SaD0+fIbssWb22GViB2PQZ0crvM1r8LElLPOOIdCEyxZ87bL2omDQLCmicQ28dCB0TcR/ezgjigleoIUor49ybyp2WclRK4FawFRF99CujPlqJrW8NACc0zjnZXCue5raqUA1zVOJCEg+BpWc9VD2zR8ZOG8w3PuAus=
 
+# repro2.pkgbuild.com
+repro2.pkgbuild.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE/RhEBp4KFplFM6l/oLUc1aNeQtKXi+dGmP6JUtdLNfHg6TVfDFQDVQo7gIFSTulvOybsccJeJWjYBAmPhsvFs=
+repro2.pkgbuild.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICYgWZak/Rud9WuGQNiDoSLgOOOey+6ig415au8PfI8S
+repro2.pkgbuild.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIKntcMeZrr1qi4oQx5BKFpsZQpxszAOH9bX/k1Xv2Owf7mmJscYXniyOG0PQlQrzUhFdQ1ZyI2RVXyqWboSwjq2TZNu7t41AUelB1WW9WCCRtULxKeskj6I+FTIWmNweBf4JK1VxmaFxceuN1KZsIxzdw0TyZB0Ub52RD7xNdtc65cvFV9wbLVmXYLORko0rIJcLxwuyPDkjUoap6kLGUULyPzmU92RuE8A+flPVg4lctBWJu2o18os/611qbaZUqL2mQl0ngJYSGwf1NXb5QHhRQw26HcO2OdJONMRGxlSBTfNmy0QmTFY7i7jFF09bhtHcsOnEhdrBoyH1VAYc2Ltr/gC61gOKhFGvtkMg5DJCRjiQ1h+CQqMXY9kzHfHQ1QxmfyMHkBgSB6Y0JPqc0eiaUWx+HYvMWpSme67mO5z0DodQcMVAK4YLT4s9LOqMGtuEtr/psW/1OVJoIJ5JpKeBKgX1605uGkiqw7ieXL+a/77ij2DzNRykW0hhLgwM=
+
 # reproducible.archlinux.org
 reproducible.archlinux.org ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCs2GNtPdFkb9iiclsKjMvvTygCj1up0rKqhTYSC8GFFWgZtE1lsNjSEiFm9fQ0ihYciUfLWh9TlIsQpmVE33zg=
 reproducible.archlinux.org ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDjL6uJx0faep1bFH6WdydUJ5YvLAoYpbwXqP7FAYA3y
 reproducible.archlinux.org ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/kp2Rm2jhGQn0PJskhrmxUkYDDeEC8tXwvR+J+t40KACQnqi4SMdeu61Jyy2NBdBOw3muegb9kspOzN+NSQFpjYhfkYJGswyF46vTkgXUWJpuHBKiWkWmZzVKcveS6iAOEDLaeAesRHmcuEjaK323d0QDf6vJ4Gbb2veFoW5Kip37phrjZH2RCCLFdrw1IGopPuKHYDGwIIqt0hQLSkyq/A1BgmiR+eo4S8AlN7D/w0MVz7bKIuQkSmm5taTjo9FpOJJ0O6KMFXrXCPKPDuzSa/zMSQgNw+JnwhjQRbnH5AUB+xFe/qt8wstBGE/VT7WoyOsfTvfsgdQmz55BgPxR9lZI+rKl43Lp3zGojCaVOd+9wV08DgGN13W7imGqWuVq0vPyIkGl19Ch5mgemED49g2ehddeAUt0FR/MM8ewD1U4cHYpd4PHLgOdC9KfGvRJkymWFPd/EypwhEdFSCJ4BNDbQfYl4ciXnuTmbh3eM/Z/TjAmztQqAok08KAQYd8=
 
+# runner1.archlinux.org
+runner1.archlinux.org ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFXWZCrWgRV7mPbjSLccWOlq7W0TwWyeVc/BWwFMstmBFwEDwo89VseT9zfaabHgPjLmTaN+uzQg75xNIzAJl2U=
+runner1.archlinux.org ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMO4pNVClWcKyQXI8oqhqDWKxYkVu+/1qUIz6vkVG2St
+runner1.archlinux.org ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDepdBpbpetW7NjMX5H3r6ZJj6lbde3p5RswTM80pazzOacXaRH8SHo0acuyabsJY0LJxgXzq9wZXvVasQ4L2zV+byPXvvT8CoyPtgeknk53beByNNvO2PUPuidtXhVspDAhqL/LNxUUBbel1ATRx+BX3cUo71A+NoFT3p6I5zIsUWbBfPP34Idckj8T1+54pAJ25bf133ustQfoOU0pZs+JsJEnFpMK8qrLk5Pt0OcvvQ9/42OI7ZSYg/mRaRX0JYQCAKyAIhSlrCynrpmj+wh38YnbT/fB5uBt6Gv80rC5e7lvRBy09MXK35YeN8Jr7rXLedH6XGBCf0IrcS1bp+JkfXPjMe9uDSQyK4ujElrYcVZlbrt4OyIh0OGgShmOW+pwouerypHuSzzjeDH8kl0UUa+ZijzQvIrm861EbHmusBVfoLJBVUQk/bPpUh+Zzu0cANPDEE6RjzUsFRReAga4NIqscOy7/YNNc4aAVyVLgFEquNBvUKsXWJoY08PTDE=
+
 # runner2.archlinux.org
 runner2.archlinux.org ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBUZk2jW9uO9Hnjmbf9Bnpm2/HrR5vg8wpVPIUgcSQhkoU7B54j28UgVvnmcEeoD9gyO9gH9gJG9xN95kM3JobI=
 runner2.archlinux.org ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINIyyRcCSU/To0KfALKiFpkl1tEC3KyRInma0CpNzV21

host_vars/america.mirror.pkgbuild.com

+hostname: "america.mirror.pkgbuild.com"
+ipv4_address: "143.244.34.62"
+ipv4_netmask: "/25"
+ipv4_gateway: "143.244.34.126"
+filesystem: "btrfs"
+network_interface: "en*"
+system_disks:
+  - /dev/sda
+  - /dev/sdb
+  - /dev/sdc
+raid_level: "raid5"

host_vars/asia.mirror.pkgbuild.com

+hostname: "archive1.mirror.pkgbuild.com"
+
+ipv4_address: "84.17.57.98"
+ipv4_netmask: "/24"
+ipv4_gateway: "84.17.57.110"
+filesystem: "btrfs"
+network_interface: "en*"
+system_disks:
+  - /dev/sda
+  - /dev/sdb
+  - /dev/sdc
+raid_level: "raid5"

host_vars/europe.mirror.pkgbuild.com

+hostname: "europe.mirror.pkgbuild.com"
+
+ipv4_address: "89.187.191.12"
+ipv4_netmask: "/26"
+ipv4_gateway: "89.187.191.62"
+filesystem: "btrfs"
+network_interface: "en*"
+system_disks:
+  - /dev/sda
+  - /dev/sdb
+  - /dev/sdc
+raid_level: "raid5"

host_vars/repro2.pkgbuild.com

+hostname: "repro2.pkgbuild.com"
+
+ipv4_address: "212.102.38.209"
+ipv4_netmask: "/24"
+ipv4_gateway: "212.102.38.222"
+
+filesystem: "btrfs"
+network_interface: "en*"
+
+system_disks:
+  - /dev/sda
+  - /dev/sdb
+raid_level: "raid1"
+
+rebuilderd_workers:
+ - repro21
+ - repro22
+ - repro23
+ - repro24

host_vars/runner1.archlinux.org

+hostname: "runner1.archlinux.org"
+
+ipv4_address: "84.17.49.250"
+ipv4_netmask: "/24"
+ipv4_gateway: "84.17.49.254"
+filesystem: "btrfs"
+network_interface: "en*"
+system_disks:
+  - /dev/sda
+  - /dev/sdb
+raid_level: "raid1"
+
+configure_network: true

hosts

@@ -18,6 +18,11 @@ repro1.pkgbuild.com
 [mirrors]
 mirror.pkgbuild.com
 
+[archive_mirrors]
+asia.mirror.pkgbuild.com
+america.mirror.pkgbuild.com
+europe.mirror.pkgbuild.com
+
 [borg_clients]
 apollo.archlinux.org
 luna.archlinux.org
@@ -69,6 +74,7 @@ aur-dev.archlinux.org
 dragon.archlinux.org
 
 [gitlab_runners]
+runner1.archlinux.org
 runner2.archlinux.org
 secure-runner1.archlinux.org
 
@@ -78,6 +84,10 @@ repro1.pkgbuild.com
 [rebuilderd]
 reproducible.archlinux.org
 
+[rebuilderd_workers]
+repro1.pkgbuild.com
+repro2.pkgbuild.com
+
 [memcached]
 aur.archlinux.org
 aur-dev.archlinux.org
@@ -107,3 +117,10 @@ bugs.archlinux.org
 gemini.archlinux.org
 luna.archlinux.org
 repro1.pkgbuild.com
+
+[kape_servers]
+asia.mirror.pkgbuild.com
+america.mirror.pkgbuild.com
+europe.mirror.pkgbuild.com
+repro2.pkgbuild.com
+runner1.archlinux.org

playbooks/archive-mirrors.yml

@@ -5,6 +5,8 @@
   roles:
     - { role: common }
     - { role: tools }
+    - { role: firewalld }
+    - { role: hardening }
     - { role: sshd }
     - { role: root_ssh }
     - { role: certbot }

playbooks/repro1.pkgbuild.com.yml → playbooks/rebuilderd-workers.yml

 ---
 
-- name: setup repro1.pkgbuild.com
-  hosts: repro1.pkgbuild.com
+- name: common playbook for rebuilderd_workers
+  hosts: rebuilderd_workers
   remote_user: root
   roles:
     - { role: common }

tf-stage1/archlinux.tf

@@ -131,6 +131,27 @@ resource "hetznerdns_record" "pkgbuild_com_mirror_aaaa" {
   type    = "AAAA"
 }
 
+resource "hetznerdns_record" "pkgbuild_com_america_a" {
+  zone_id = hetznerdns_zone.pkgbuild.id
+  name    = "america.mirror"
+  value   = "143.244.34.62"
+  type    = "A"
+}
+
+resource "hetznerdns_record" "pkgbuild_com_asia_a" {
+  zone_id = hetznerdns_zone.pkgbuild.id
+  name    = "asia.mirror"
+  value   = "84.17.57.98"
+  type    = "A"
+}
+
+resource "hetznerdns_record" "pkgbuild_com_europe_a" {
+  zone_id = hetznerdns_zone.pkgbuild.id
+  name    = "europe.mirror"
+  value   = "89.187.191.12"
+  type    = "A"
+}
+
 resource "hetznerdns_record" "pkgbuild_com_repro1_a" {
   zone_id = hetznerdns_zone.pkgbuild.id
   name    = "repro1"
@@ -145,6 +166,13 @@ resource "hetznerdns_record" "pkgbuild_com_repro1_aaaa" {
   type    = "AAAA"
 }
 
+resource "hetznerdns_record" "pkgbuild_com_repro2_a" {
+  zone_id = hetznerdns_zone.pkgbuild.id
+  name    = "repro2"
+  value   = "212.102.38.209"
+  type    = "A"
+}
+
 resource "hetznerdns_record" "pkgbuild_com_www_a" {
   zone_id = hetznerdns_zone.pkgbuild.id
   name    = "www"
@@ -600,6 +628,13 @@ resource "hetznerdns_record" "archlinux_org_reproducible_a" {
   type    = "A"
 }
 
+resource "hetznerdns_record" "archlinux_org_runner1_a" {
+  zone_id = hetznerdns_zone.archlinux.id
+  name    = "runner1"
+  value   = "84.17.49.250"
+  type    = "A"
+}
+
 resource "hetznerdns_record" "archlinux_org_runner2_a" {
   zone_id = hetznerdns_zone.archlinux.id
   name    = "runner2"