Skip to content

Disallow scraping the search results on the AUR

Jelle van der Waa requested to merge aur_robots.txt into master

As the queries for searching are the most 'heavy' queries we can disallow bots to scrape through all search results. Archweb uses the same robots.txt rules for disallowing indexing search results.

Merge request reports

Loading