After=network-online.target is likely insufficient by itself.

Fixes: 104d08a1 ("archweb: Put most services after network-online.target")
Fixes: c844d0cb ("Split storage box monitoring into new text collector")

We noticed readlinks and reporead on gemini failing to connect to the
archweb database immediately after rebooting. To fix this, place them
after network-online.target. Do the same for all but one of the other
service units even though they run on timers 10-15 minutes after boot
for completeness and correctness.

Fix #177

Reverts caching issue

Add a default rate limit for 20 req/s for the uwsgi endpoint and
automatically ban users who reach this limit. The nginx-limit-req rule
does not ban users who reach the rss limit as these are not likely DoS
attempts.

Thorben Günther authored 3 years ago and Kristian Klausen committed 3 years ago

Closes #358

Fix deploying archweb on gemini

We did the revert to help a small business, but they had some time to
fix their setup, so we can revert the revert.

This reverts commit a842fbe7.

This implements authentication to our repos.archlinux.org tier 0 mirror
via archweb.

Fixes: 316b8517 ("Add missing "create ssl cert" tasks")

The latter can cause duplicate Content-Type headers. Thanks to strcat
for notifying me of the issue.

This adds Last-Modified header support so we should have less cpu spikes
for the archweb-rebuilderd service.

This is causing issues for a small business, which can't reach their
"remote systems" anymore due to NM reporting "limited access".
We should be able to revert this in 1-2 weeks.

This reverts commit b909fa58.

Kristian Klausen authored 3 years ago and Jelle van der Waa committed 3 years ago

A extra access_log entry was added with the following commands:
$ cd roles
$ grep -lr access_log | xargs -P 1 -n 1 sed -i '/access_log/ s/\(.*\)\( \)\(\(reduced\|main\);$\)/\1 \3\n\1.json json_\3/'

Kristian Klausen authored 4 years ago and Jelle van der Waa committed 3 years ago

[1] https://github.com/yandex/gixy/blob/641060d6355fbb5bd71695928a2bf14a9bcb8bf2/docs/en/plugins/aliastraversal.md

Fix #291

With the ping.al.org domain added in[1] and a updated networkmanager
package shipped[2], we can now remove the file from the main domain.

[1] 498d5304 ("Merge branch 'ping' into 'master'")
    fabccd0f (""Move" NM connectivity check file to a subdomain")
[2] https://github.com/archlinux/svntogit-packages/commit/fb573170fc63c18f64eb4f155ed64a966a61b037

Cache urls which urls marks as can be cached by nginx. This offloads our
uwsgi workers and allows for speedier delivery of RSS feeds and other
cached routes.

Due to users misconfiguring their conky to query for rss updates every
second add proper rate limitting to all rss endpoints in nginx.

yaml: truthy value should be one of [false, true] (truthy)
yaml: wrong indentation: expected 4 but found 2 (indentation)
yaml: too few spaces before comment (comments)
yaml: missing starting space in comment (comments)
yaml: too many blank lines (1 > 0) (empty-lines)
yaml: too many spaces after colon (colons)
yaml: comment not indented like content (comments-indentation)
yaml: no new line character at the end of file (new-line-at-end-of-file)
load-failure: Failed to load or parse file
parser-error: couldn't resolve module/action 'hosts'. This often indicates a misspelling, missing collection, or incorrect module path.

Fixes announcement release email.

The former approach to export a maildir and iterate over it with a
script broke when the mail server and the web server got on their
own hosts. This will use IMAP IDLE to check for new mails and pass
them instantly to the djange manage.py script without storing the mail
locally.