Add robots.txt
Currently the website has no robots.txt and I think we want to exclude /search from being indexed. (Archweb does something similiar)
Due to an influx of spam, we have had to temporarily disable account registrations. Please write an email to accountsupport@archlinux.org, with your desired username, if you want to get access. Sorry for the inconvenience.
Currently the website has no robots.txt and I think we want to exclude /search from being indexed. (Archweb does something similiar)
assigned to @lahwaacz
added scopeenhancement label
Hmm, I don't see robots.txt
in archweb, but https://archlinux.org/robots.txt apparently exists... There are also some tags in the package search view.
mentioned in merge request infrastructure!470 (merged)
mentioned in commit infrastructure@43772844
closed with merge request infrastructure!470 (merged)