Add Mumble server

As per my announcement to arch-devops[1] and staff, this adds a Mumble
server for Arch Linux.

The password for the special root user SuperAdmin is automatically
generated on first launch and printed to the logs. I went ahead and
added it to the vault. It should not usually be required to login as
SuperAdmin though as long as there are user admins around.

This uses certbot for local certificates.

[1] https://lists.archlinux.org/archives/list/arch-devops@lists.archlinux.org/thread/AHAOSTGFJTLQDSXLWFORDKGR6RDVHYEI/

docs/servers.md

@@ -118,6 +118,11 @@ Medium-fast-ish Equinix Metal Arch Linux box.
   - [Grafana](https://monitoring.archlinux.org) and [docs/grafana.md](./docs/grafana.md)
   - Prometheus
 
+## mumble.archlinux.org
+
+### Services
+  - Mumble
+
 ## dashboards.archlinux.org
 
 Prometheus, and Grafana server which receives selected performance/metrics from monitoring.archlinux.org and make them public accessible.
@@ -162,4 +167,4 @@ The [Arch Linux Archive](https://archive.archlinux.org) is mirrored to three ded
 ## gitlab.archlinux.org
 
 ### Services
-- Gitlab
\ No newline at end of file
+  - GitLab

docs/ssh-hostkeys.txt

@@ -205,6 +205,15 @@
 256 MD5:fe:a1:ab:4d:f6:5d:76:f9:a3:99:be:fd:51:ee:77:ed root@archlinux-packer (ED25519)
 3072 MD5:ad:ee:a6:6d:b7:9b:f0:f7:78:9f:df:b4:53:2e:5f:9f root@archlinux-packer (RSA)
 
+# mumble.archlinux.org
+256 SHA256:+Kb9ZYX3TBuzq0zsenFFxCkP4V72a6sn6GNt6iPZaoo root@archlinux-packer (ECDSA)
+256 SHA256:emrNzCZ+aasNz8C6kcDl/jPYWgqDq4Yl4Epzvw3KPc4 root@archlinux-packer (ED25519)
+3072 SHA256:VCqfjI+1rtVXQNkEK2Tk3Sj6iIHlB0jfFGKXt0T+kUA root@archlinux-packer (RSA)
+
+256 MD5:7a:96:1c:78:49:5d:e6:79:89:e8:c3:41:cc:cb:86:04 root@archlinux-packer (ECDSA)
+256 MD5:a7:3c:5a:11:e8:35:7c:6d:7e:4f:1c:69:2f:27:02:6f root@archlinux-packer (ED25519)
+3072 MD5:36:0e:0b:00:ca:ea:e9:70:f8:00:96:0c:63:e1:0c:19 root@archlinux-packer (RSA)
+
 # opensearch.archlinux.org
 256 SHA256:Fq62NmjmKfqHPvXk4t983pikezNWbGUokYoGljjTRlo root@archlinux-packer (ECDSA)
 256 SHA256:9BrCmtZiltz907mhTMA/5UVxy1Uwjmb+eN5yjbcVt2c root@archlinux-packer (ED25519)

docs/ssh-known_hosts.txt

@@ -125,6 +125,11 @@ monitoring.archlinux.org ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAA
 monitoring.archlinux.org ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJCU4tNW4WHTQ43+HBbho/sbsU3BCzildSOziaJrVNvE
 monitoring.archlinux.org ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDVAMU3iku88nPDAKjB++je4RRRkotwNdJEhRcO45Ujslhbq67D6BwcnaliR0ekZuhkQFs13dTNVGeb1VqN3I/wHVaECsd/Gz7Q2M5Ki2CqdUR8ztGaW/eWpY9r8Yk+h/fWdnZdnJPYhk7uZftJI9buqyqpkthvjQy9fZ2wyOb/BAk+7BYUdclcvCEMlW9HQljpgmj7snjTpMYMN0t3U7X3xydcOO6PwNIoSikufuMmbtCqtsUx/Xl1mVU2Xi584L8arjoKn9a4OjMUDorqAlFLeco6bWn5XEdfim6e+W55ZKg333j4KGMBFVW5Dk5mZGKfykalq4WONMe3nu0m4EqYFA/rGG/smliqjxCbWu9N6eDw1gKYOeq5gzx7ppQ9zL3BjL3gl+AbeUckxNCQ+zM66amZC6GmciiMq+hnpqeTUhocaGeriGVda4vO+IlCp4Wwx1zqcCZaHyzt/eIWT9DuXDqHq4gAshluGUR0gFTJ/0qhrYxQA/dW681LE3r9YLE=
 
+# mumble.archlinux.org
+mumble.archlinux.org ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGPCQmC4yI3bfvzAd4RgFn+EI4qcsBa3TcneSJSoMjADfvYaWMB3yIJ0LWc1LkSpJVMF7kAS8F16pdOwXJPo6xk=
+mumble.archlinux.org ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKq0F4chCcISD1B+uYNjH/zTSaHp76is2n6YBQ7HYiLf
+mumble.archlinux.org ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8gS2KkTmfjn7915xPAVe6xeOD7ZicQhzmZHmV0h3zt5hwBn9PQ1G82QplC5ZyFzxGBnHU/2mDFL+6uPLptVVdbSbGYbRlYQtFRnWjUgrzC4GX5JnNnJpTNF9SuUwfhgCZFVmuAwEl5AI6K7N/A8ox5YyT6OxY6lz54zpmn1s2S2zOMGvm4BJxiCFMxCmZKk37DLZ7fz05REajsxaqZ7Otz6MrbSOXpD0EdtY1APA2vRbw1N9GTpJYBmOeu8wv43B/+/20vtJJ+mVkdBRFiRG0NdhfaZ2qYeco3HSgb7EBp5mI8HeWBdqPuCfqxiMqyMmfWVM+PrsC2Qn47nZcSpWojwX9pufN/GQ/IiUM3nUxbRhMo8lx+qEChqMz2BzKEoyLtEfb3FQM59CPiXjZs8gALgmxj1GUZh2LiHjLxZ7NNevIWt0rYG7hEdAh4QVEHa3vKwWUjDOZ8v7coI+tmqEdNfcQQptOvoIO9hdMUPafkxmZbAU3d/ejB7fLnXJsGiE=
+
 # opensearch.archlinux.org
 opensearch.archlinux.org ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPfEiVTq6bLKydE0yse2kiw5Tznz3Kb+Du92HCg61EeFQs/TzOuo4vKZCr3Rt7/6bV2aMZU8HXE0223AukEH4aU=
 opensearch.archlinux.org ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKom1E2rOlhSY7b4Cd+L6IpAjZWA2yIX4/ndeENRbn9c

group_vars/all/dyn_dns.yml

@@ -12,3 +12,8 @@ dyn_dns_zones:
     allowed_ipv6: "{{ groups['gitlab_runners'] | map('extract', hostvars, ['ipv6_address']) }}"
     valid_qtypes: [A, AAAA]
     subdomains: only
+  _acme-challenge.mumble.archlinux.org:
+    key: mumble
+    allowed_ipv4: "{{ [hostvars['mumble.archlinux.org']['ipv4_address']] }}"
+    allowed_ipv6: "{{ [hostvars['mumble.archlinux.org']['ipv6_address']] }}"
+    valid_qtypes: [TXT]

group_vars/all/vault_dyn_dns_keys.yml

 $ANSIBLE_VAULT;1.1;AES256
-62393237353533363738376335336564623464336332393733306465333339376130613338356537
-6166666538303939313238323238616433653036376662360a323663613934636539333365303166
-33343266613234363965363233666165383333343862326436313935636631326266363462613033
-3937393135656534370a663035633362643931653864336336396535373038396165633934366433
-31656663396538376337373762386162386665353639336235363233643139303763333861376339
-62306130363039376431396234333030616235306530343336326237656638636435363038663931
-39356535643265616337306530393962373537336335333764363565313939373565326561613066
-36633931656662393538353836353365386634663736356131323435333265653832656162306230
-64326535353532373137656535386531333536353531643863646135386664333030363564376463
-61386537306235356666353761383237336133376665393365663636386238373534623833306430
-37323336623537613034643763363439643063633433323431623932646465363230316533356337
-34623964653036383766316336373462363562333963663939333431643665643737643164396565
-38396332356630366665666239656562313430363432366639373235343430653236356438643131
-65623438313963356630333939636663393539656463376339326631636263313564636432343635
-39656466323965626264623332393630333035396638653039343536373337643165313564333363
-36626239303836383932336537313061663961636137396162303838356661386636303262653633
-33336665306634363866386237623733643663313136373037376631363364343161373731626637
-30346433666230663564643731616566663339393166343061333033386462366663383839653631
-363865646464333236663262323265376363
+39316235626337313266636565363065336436373337353935633566303635323366336266363632
+3765653337333964376366383263323566333765356336610a366431326163383737333634303833
+66333963336137323866356433306366353362623230336465633962306134393237323363626530
+3335633834356232330a613764613230353564356238616331623131346431373665383332663332
+37643934373831373066303532356263336631353262326132373738643564333631386336343930
+65323065386365346637373235656232356137646237643730316437393962376632656333313864
+36383062626462616563623431363466343263623161623531323136376161336632356439636666
+63383738313233336331393739316166383565343134343031353063383231636132653264633435
+38623661613036353034363737623330313234313764326538616439336661393666656238633662
+33613765353131636262623431323037313633343030646165626139373234343461373965396331
+31333466316434613539323561336562616637666134323630616164653433353938363666383333
+64383265323630306165613965353563643038313835306365353931653461656430383532383962
+32356636333461326135383364366235366561613366646133313033653637626161663934616532
+61663237633966613935626635346463613836653734373331363135313066666262323762613039
+37353033373966323539653231303633383764656565646166323762316634616236346538313565
+33613830353633646664643232346534656337376161373063626134343162616562313566346230
+66326339633564346564393834383131316336346539653264346431323436656137626635613162
+61626166656364386330326335323738643062356532343635343730313565656334303637303636
+35316232333432653236623932386661306336353465333833626330643239393861303165666331
+62636338386132303366663437393832353637626362303635306136353962363664353266656330
+66373431313434333666653930346135623231363364626434633235653938393231653761376336
+31393763343032623664666662366235353237366531626666646264326566303335393834336262
+34316631303833346166306165356564666232373265366338663961313865613065366362636533
+32366463316430653463373163376335396636616234306562363832323437636362316562623135
+65626563633666623462653630306531326135353037313133653562306638353331

group_vars/all/vault_mumble_server.yml

+$ANSIBLE_VAULT;1.1;AES256
+34323763363030343563626539633432393766383164346164343534343930356664333863343938
+3730346635306563383762373464633165356637373764640a633031646165333933623633366136
+61613733623735633337626134633266393464666465363065343039653666336565313638386538
+6235626535343035660a633435626433353666386463346464653833326131653437613637386363
+65383534306234333535633834623562316137353563366565653439343662613839393162613765
+32616335303436653637343439373634303533373265313062653630646333326661613936633438
+34313964636637653431333237306664666436633239366461343936316438363066623439356463
+33393833653737353262366566613737633761383537633266343561636562336330653033313761
+31316234336463396566366264383033376537336231313962643831626437316639

group_vars/geo_mirrors/misc.yml

 certbot_dns_support: true
+certbot_tsig_name: certbot
 geo_mirror_domain: geo.mirror.pkgbuild.com

host_vars/mumble.archlinux.org/misc

+filesystem: btrfs
+
+ipv4_address: "188.245.228.0"
+ipv4_netmask: "/32"
+ipv6_address: "2a01:4f8:c012:d0ce::1"
+fail2ban_jails:
+  sshd: true
+  postfix: false
+  dovecot: false
+  nginx_limit_req: false
+wireguard_address: 10.0.0.46
+wireguard_public_key: jiA9adrFKJuZsxS1DMHi+gkb4iWj3w0CNGWY/elxpzk=
+certbot_dns_support: true
+certbot_tsig_name: mumble

host_vars/mumble.archlinux.org/vault_wireguard.yml

+$ANSIBLE_VAULT;1.1;AES256
+30613530316630386565666462353635333163343337383639346132366562616533323036633433
+3131353639386564353062626639313937333661323535610a353463353866303962333230633632
+64316664643431616537396233363730333332633134376661633137643135366461643531626363
+6435613738396132650a353130653335373630356336613339363463313562323962373833363831
+32663166366135323939386336663061356637616364636439323430633837616534663139396562
+62333964613937623763646637346136363638613138366335383765376131666536363539353938
+34653030393432373666663934386439396135346532373739333838373036326531656635663532
+64306330643130663936

hosts

@@ -46,6 +46,7 @@ mail.archlinux.org
 matrix.archlinux.org
 md.archlinux.org
 monitoring.archlinux.org
+mumble.archlinux.org
 phrik.archlinux.org
 quassel.archlinux.org
 reproducible.archlinux.org
@@ -122,6 +123,7 @@ matrix.archlinux.org
 md.archlinux.org
 mirror.pkgbuild.com
 monitoring.archlinux.org
+mumble.archlinux.org
 opensearch.archlinux.org
 phrik.archlinux.org
 quassel.archlinux.org

playbooks/mumble.archlinux.org.yml

+- name: Setup mumble server
+  hosts: mumble.archlinux.org
+  remote_user: root
+  roles:
+    - { role: firewalld }
+    - { role: wireguard }
+    - { role: hardening }
+    - { role: common }
+    - { role: sshd }
+    - { role: root_ssh }
+    - { role: borg_client, tags: ["borg"] }
+    - { role: prometheus_exporters }
+    - { role: promtail }
+    - { role: fail2ban }
+    - { role: certbot }
+    - { role: mumble_server }

roles/certbot/templates/rfc2136.ini.j2

 dns_rfc2136_server = {{ dyn_dns_server }}
-dns_rfc2136_name = certbot
-dns_rfc2136_secret = {{ dyn_dns_keys['certbot'].secret }}
-dns_rfc2136_algorithm = {{ dyn_dns_keys['certbot'].algorithm | upper }}
+dns_rfc2136_name = {{ certbot_tsig_name }}
+dns_rfc2136_secret = {{ dyn_dns_keys[certbot_tsig_name].secret }}
+dns_rfc2136_algorithm = {{ dyn_dns_keys[certbot_tsig_name].algorithm | upper }}

roles/mumble_server/files/restart-mumble-server.sh

+#!/bin/bash
+if [[ "$1" == "renew" ]]; then
+  systemctl restart mumble-server
+elif [[ "$1" == "post" ]]; then
+  install -v -o _mumble-server -g _mumble-server -m 640 /etc/letsencrypt/live/mumble.archlinux.org/cert.pem /var/lib/mumble-server/cert.pem
+  install -v -o _mumble-server -g _mumble-server -m 640 /etc/letsencrypt/live/mumble.archlinux.org/privkey.pem /var/lib/mumble-server/privkey.pem
+  install -v -o _mumble-server -g _mumble-server -m 640 /etc/letsencrypt/live/mumble.archlinux.org/fullchain.pem /var/lib/mumble-server/fullchain.pem
+fi

roles/mumble_server/handlers/main.yml

+- name: Restart mumble-server
+  service: name=mumble-server state=restarted

roles/mumble_server/tasks/main.yml

+- name: Install mumble-server
+  pacman: name=mumble-server state=present
+
+- name: Open firewall holes
+  ansible.posix.firewalld: port={{ item }} permanent=true state=enabled immediate=yes
+  when: configure_firewall
+  with_items:
+    - "64738/tcp"
+    - "64738/udp"
+  tags:
+    - firewall
+
+- name: Configure mumble-server
+  template: src=mumble-server.ini.j2 dest=/etc/mumble/mumble-server.ini owner=root group=root mode=0644
+  notify:
+    - Restart mumble-server
+
+- name: Add certbot hook
+  copy: src=restart-mumble-server.sh dest=/etc/letsencrypt/hook.d/restart-mumble-server.sh owner=root group=root mode=0755
+
+- name: Create ssl cert for mumble-server
+  include_role:
+    name: certificate
+  vars:
+    domains: ["{{ inventory_hostname }}"]
+    challenge: "DNS-01"
+  register: result
+
+- name: Install the certificate by running the certbot hook
+  command: /etc/letsencrypt/hook.d/restart-mumble-server.sh post
+  args:
+    creates: /var/lib/mumble-server/fullchain.pem
+
+- name: Enable and start mumble-server.service
+  service: name=mumble-server enabled=yes state=started

roles/mumble_server/templates/mumble-server.ini.j2

+; See https://github.com/mumble-voip/mumble/blob/master/auxiliary_files/mumble-server.ini
+; for all values and explanations.
+
+; Path to database. If blank, will search for
+; mumble-server.sqlite in default locations or create it if not found.
+database=/var/lib/mumble-server/mumble-server.sqlite
+
+; Specifies the file the server should log to. By default the server
+; logs to the file 'mumble-server.log'. If you leave this field blank
+; on Unix-like systems, the server will force itself into foreground
+; mode which logs to the console.
+logfile=
+
+; Welcome message sent to clients when they connect.
+; If the welcome message is set to an empty string,
+; no welcome message will be sent to clients.
+welcometext="<br />Welcome to <b>Arch Linux</b>.<br />Enjoy your stay!<br />"
+
+; Port to bind TCP and UDP sockets to.
+port=64738
+
+; Specific IP or hostname to bind to.
+; If this is left blank (default), the server will bind to all available addresses.
+;host=
+
+; Password to join server.
+serverpassword="{{ vault_mumble_server_password }}"
+
+; Maximum bandwidth (in bits per second) clients are allowed
+; to send speech at.
+bandwidth=558000
+
+; Maximum number of concurrent clients allowed.
+users=100
+
+; These two settings allow to configure the per-user rate limiter for some
+; command messages sent from the client to the server. The messageburst setting
+; specifies an amount of messages which are allowed in short bursts. The
+; messagelimit setting specifies the number of messages per second allowed over
+; a longer period. If a user hits the rate limit, his packages are then ignored
+; for some time. Both of these settings have a minimum of 1 as setting either to
+; 0 could render the server unusable.
+messageburst=5
+messagelimit=1
+
+; Respond to UDP ping packets.
+;
+; Setting to true exposes the current user count, the maximum user count, and
+; the server's maximum bandwidth per client to unauthenticated users. In the
+; Mumble client, this information is shown in the Connect dialog.
+allowping=true
+
+; You can set this setting to a channel ID, and the user will automatically be
+; moved into that channel instead. Note that this is the numeric ID of the
+; channel, which can be a little tricky to get (you'll either need to use an
+; RPC mechanism, watch the console of a debug client, or root around through
+; the server database to get it).
+;
+defaultchannel=5
+
+; When a user connects to a server they've already been on, by default the
+; server will remember the last channel they were in and move them to it
+; automatically. Toggling this setting to false will disable that feature.
+;
+;rememberchannel=true
+
+; How many seconds should the server remember the last channel of a user.
+; Set to 0 (default) to remember forever. This option has no effect if
+; rememberchannel is set to false.
+;rememberchannelduration=0
+
+; Maximum length of text messages in characters. 0 for no limit.
+;textmessagelength=5000
+
+; Maximum length of text messages in characters, with image data. 0 for no limit.
+imagemessagelength=512000
+
+; Allow clients to use HTML in messages, user comments and channel descriptions?
+allowhtml=true
+
+; If you have a proper SSL certificate, you can provide the filenames here.
+; Otherwise, the server will create its own certificate automatically.
+sslCert=/var/lib/mumble-server/cert.pem
+sslKey=/var/lib/mumble-server/privkey.pem
+sslCA=/var/lib/mumble-server/fullchain.pem
+
+; By default, in log files and in the user status window for privileged users,
+; Mumble will show IP addresses - in some situations you may find this unwanted
+; behavior. If obfuscate is set to true, the server will randomize the IP addresses
+; of connecting users.
+;
+; The obfuscate function only affects the log file and DOES NOT effect the user
+; information section in the client window.
+obfuscate=true
+
+; A flag dictating whether clients may use the built-in recording function. Newer
+; clients will respect this option in the UI (e.g. disable the recording feature
+; in the UI). Additionally any client that tries to start a recording is kicked
+; from the server with a corresponding message, if recording is disabled.
+; Default is true. This option was introduced with Mumble server 1.5.0.
+;
+; allowRecording=true
+
+; You can configure any of the configuration options for Ice here. We recommend
+; leave the defaults as they are.
+; Please note that this section has to be last in the configuration file.
+;
+[Ice]
+Ice.Warn.UnknownProperties=1
+Ice.MessageSizeMax=65536

roles/prometheus/defaults/main.yml

@@ -90,6 +90,7 @@ blackbox_targets:
   tls_connect:
     - mail.archlinux.org:465
     - mail.archlinux.org:993
+    - mumble.archlinux.org:64738
     - coc.archlinux.org:443
     - git.archlinux.org:443
     - rsync.archlinux.org:443

tf-stage1/archlinux.tf

@@ -125,6 +125,10 @@ locals {
       server_type = "cx32"
       domain      = "monitoring"
     }
+    "mumble.archlinux.org" = {
+      server_type = "cx22"
+      domain      = "mumble"
+    }
     "opensearch.archlinux.org" = {
       server_type = "cx22"
       domain      = "opensearch"
@@ -622,6 +626,14 @@ resource "hetznerdns_record" "archlinux_org_origin_ns1" {
   ttl     = 86400
 }
 
+resource "hetznerdns_record" "archlinux_org_acme_challenge_mumble_ns1" {
+  zone_id = hetznerdns_zone.archlinux.id
+  name    = "_acme-challenge.mumble"
+  value   = "redirect.archlinux.org."
+  type    = "NS"
+  ttl     = 86400
+}
+
 # TODO: Commented currently as we have no idea how to handle SOA stuff with Terraform:
 # https://github.com/timohirt/terraform-provider-hetznerdns/issues/20
 # https://gitlab.archlinux.org/archlinux/infrastructure/-/merge_requests/62#note_4040