Merge branch 'add-robots' into 'master'

archweb: Add robots.txt

Closes #358

See merge request archlinux/infrastructure!452
2 jobs for master in 46 seconds (queued for 5 seconds)