Optimise the AURWeb

The server is constantly busy with quite some mariadb load:

• Are there more things we can cache?
• We are getting scraped a lot now by LLM's, what do they scape, package pages?
• Can we cache the RSS feeds in nginx? Archweb does this for example
• Find the top requests of AURWeb (we should have some prometheus stats from aurweb
• Lower requests / sec, it is currently quite high (20 req/s), surely a normal page load is like 1-3 requests?
• Do we have a proper robots.txt setup for scrapers like LLM's? Crawl-Delay?
• Make more use of redis in aurweb
• Cache the recent packages in the homepage
• Can we figure out the top requests from prometheus? https://monitoring.archlinux.org/d/gmvZGXjGk/aur-statistics?orgId=1