aurweb: Create robots.txt
This should i.e. forbid crawlers to index all of the git history pages which put's unneccessary load on the server.
Link: #610 Signed-off-by: Christian Heusel christian@heusel.eu
Due to an influx of spam, we have had to temporarily disable account registrations. Please write an email to accountsupport@archlinux.org, with your desired username, if you want to get access. Sorry for the inconvenience.
This should i.e. forbid crawlers to index all of the git history pages which put's unneccessary load on the server.
Link: #610 Signed-off-by: Christian Heusel christian@heusel.eu