Fix deploying archweb on gemini

We did the revert to help a small business, but they had some time to
fix their setup, so we can revert the revert.

This reverts commit a842fbe7.

This implements authentication to our repos.archlinux.org tier 0 mirror
via archweb.

Fixes: 316b8517 ("Add missing "create ssl cert" tasks")

The latter can cause duplicate Content-Type headers. Thanks to strcat
for notifying me of the issue.

This adds Last-Modified header support so we should have less cpu spikes
for the archweb-rebuilderd service.

This is causing issues for a small business, which can't reach their
"remote systems" anymore due to NM reporting "limited access".
We should be able to revert this in 1-2 weeks.

This reverts commit b909fa58.

Kristian Klausen authored 4 years ago and Jelle van der Waa committed 3 years ago

A extra access_log entry was added with the following commands:
$ cd roles
$ grep -lr access_log | xargs -P 1 -n 1 sed -i '/access_log/ s/\(.*\)\( \)\(\(reduced\|main\);$\)/\1 \3\n\1.json json_\3/'

Kristian Klausen authored 4 years ago and Jelle van der Waa committed 3 years ago

[1] https://github.com/yandex/gixy/blob/641060d6355fbb5bd71695928a2bf14a9bcb8bf2/docs/en/plugins/aliastraversal.md

Fix #291

With the ping.al.org domain added in[1] and a updated networkmanager
package shipped[2], we can now remove the file from the main domain.

[1] 498d5304 ("Merge branch 'ping' into 'master'")
    fabccd0f (""Move" NM connectivity check file to a subdomain")
[2] https://github.com/archlinux/svntogit-packages/commit/fb573170fc63c18f64eb4f155ed64a966a61b037

Cache urls which urls marks as can be cached by nginx. This offloads our
uwsgi workers and allows for speedier delivery of RSS feeds and other
cached routes.

Due to users misconfiguring their conky to query for rss updates every
second add proper rate limitting to all rss endpoints in nginx.

yaml: truthy value should be one of [false, true] (truthy)
yaml: wrong indentation: expected 4 but found 2 (indentation)
yaml: too few spaces before comment (comments)
yaml: missing starting space in comment (comments)
yaml: too many blank lines (1 > 0) (empty-lines)
yaml: too many spaces after colon (colons)
yaml: comment not indented like content (comments-indentation)
yaml: no new line character at the end of file (new-line-at-end-of-file)
load-failure: Failed to load or parse file
parser-error: couldn't resolve module/action 'hosts'. This often indicates a misspelling, missing collection, or incorrect module path.

Fixes announcement release email.

The former approach to export a maildir and iterate over it with a
script broke when the mail server and the web server got on their
own hosts. This will use IMAP IDLE to check for new mails and pass
them instantly to the djange manage.py script without storing the mail
locally.

Giancarlo Razzolini authored 4 years ago and Jelle van der Waa committed 4 years ago

Since the network manager package is still pointing to http://www.archlinux.org for its check,
add a variable archweb_network_check_domain and change the nginx conf for both the maintenance
and regular, to allow the check to pass for the www site. Once the package is updated to point to
http://archlinux.org and after a reasonable amount of time has passed, we can remove this.

Giancarlo Razzolini authored 4 years ago and Jelle van der Waa committed 4 years ago

Changed the default domain and subsequent redirects to point to archlinux.org for everything.

Added the missing break; parameter to allow the maintenace remote machine in.

This template is very similar to the regular archweb nginx one with a few notable differences:
- Regardless of the domain, everything will go to the main domain
- It allows the network manager check to pass
- It will use the ip address of the person running the role, and exempt only that ip address from hitting the
  maintenance page. Everybody else should see the maintenance page.

Configured the variable for the custom nginx template used on maintenance mode. It is important that this template handles the maintenance
on it's own. Also, the maintenance mode was running on gemini, even though archweb_site is false there. Add a check for archweb_site, to make
sure the maintenance mode only runs on the machine hosting the site.