Merge branch 'add-robots' into 'master'

archweb: Add robots.txt

Closes #358

See merge request archlinux/infrastructure!452
2 jobs for master in 46 seconds (queued for 5 seconds)
Status Name Job ID Coverage
  Test
passed ansible-lint #28720

00:00:33

passed terraform-validate #28721

00:00:46