Disallow scraping the search results on the AUR
As the queries for searching are the most 'heavy' queries we can disallow bots to scrape through all search results. Archweb uses the same robots.txt rules for disallowing indexing search results.
Loading
Please register or sign in to comment