This should not cause any issues as MediaWiki should purge stale content
from the cache (by sending a PURGE request to the nginx-cache-purge
service).

We have used MediaWiki's file cache[2] until now, but recently the wiki
has been hammered with requests from some stupid Chinese bots/crawlers.

Caching at the web server level is faster as we avoid the PHP overhead
and it seems to make a difference (performance wise), especially when
the bots/crawlers are hitting us.

This is usual done with Varnish[3], but I went with a simple Python
service (30 LOC) for handling the PURGE requests as that is much simpler
thn adding Varnish to our stack.

[1] https://www.mediawiki.org/w/index.php?title=Manual:Performance_tuning&oldid=6670283#Page_view_caching
[2] https://www.mediawiki.org/wiki/Manual:File_cache
[3] https://www.mediawiki.org/wiki/Manual:Varnish_caching

Fix #315

This will be used for issue-bot[1][2].

[1] signstar#20 (comment 201743)
[2] https://gitlab.com/gitlab-org/distribution/issue-bot

See: https://github.com/element-hq/synapse/issues/17473

The need for UEFI booting originates from dedicated server and it does
not benefit cloud servers. It therefore makes sense to skip it on them.

The module postgresql_privs deprected the "password" parameter in favour
of the "login_password" parameter, therefore replace accordingly.

https://github.com/ansible-collections/community.postgresql/blob/main/CHANGELOG.rst#id19

Fixes #603



Signed-off-by: Christian Heusel <christian@heusel.eu>

This will be our backend for a Grafana-based APM.
It is sorely required for gaining insights into why application such as aurweb are slow.

We currently only enable the OTLP receiver as it seems to be the most
modern and best supported one.

We connect directly to the prometheus at localhost for the generated
metrics.

We're also using just storing traces locally in files instead of
something like S3.

sudo 1.9.15.p5-2 enables secure_path by default.

Signed-off-by: Christian Heusel <christian@heusel.eu>

This should i.e. forbid crawlers to index all of the git diffs which
put's unneccessary load on the server and is not really of benefit to be
indexed anyways.

Link: #610


Reviewed-by: Sven-Hendrik Haase <svenstaro@gmail.com>
Reviewed-by: Levente Polyak <anthraxx@archlinux.org>
Signed-off-by: Christian Heusel <christian@heusel.eu>

The wiki has been hammered with requests from some stupid Chinese
bots/crawlers. Adding a simple challenge (requiring a cookie to be set),
seems to be enough to throw them off.

This was initially added for all pages, but as that could affect Chinese
search engines (concern raised on the forum[1]), it was changed to only
affect "action views", which search engines are not supposed to crawl.

[1] https://bbs.archlinux.org/viewtopic.php?pid=2185963#p2185963

This will be used for installing the geoip2 module, so we can make it
more difficult for Chinese bots to crawl the wiki.

The name of the shared object file can be overridden in case it is not
named ngx_http_{{ module.name }}_module.so, e.g. srcache where the
shared object is named ngx_http_srcache_filter_module.so.

Archweb now exports Prometheus status via /metrics with request duration
information.

To shut up the linter.

Using a cert named after the primary domain with `_legacy` appended.
However, the cert is only issued for the legacy domains, not the primary
domain.

Deploy for `ipxe.archlinux.org`.

Fixes: releng#22

They might conflict with the normal configuration, so we don't want
these redirects to get cached.

With RSA 4096 instead of ECDSA.

Don't require the cert to have the same name as the first domain.

This reverts commit f5b566fa.

We no longer have cloud servers with 1-vCPU so this isn't needed.

Simplify the role by removing the configurability of "EditionIDs", and
hardcoding its value to "GeoLite2-Country GeoLite2-City". While it was
originally intended for consumers to select which database(s) to fetch,
it's not straightforward how to support multiple inclusions of the role.

This reverts commit a9b596c4.

We no longer have cloud servers with 1-vCPU so this isn't needed.

archlinux.org started rejecting connections without SNI because of
experiments with deploying HTTP/3.

See: archlinux/infrastructure!850

This is required of OCSP stapling to work, or you get warnings when
NGINX starts up:

    no resolver defined to resolve e6.o.lencr.org while requesting certificate status

Let NGINX use the local systemd-resolved as its resolver.

Fixes: archlinux/infrastructure#607

Fixes: 7fdcc769 ("Revert "dbscripts: Add tmpfiles.d/rsyncd.conf for abs and friends"")
Signed-off-by: Christian Heusel <christian@heusel.eu>

pita strudl authored 8 months ago and pita strudl committed 8 months ago

We try to ensure an atomic operation of the lastsync file. This requires creating a tmp file which needs to be ignored.
This should take care of having empty lastsync files being served.
Possible cause is that the IO is stuck thus taking several seconds to write the lastsync timestamp.
This causes mirrors to download the empty file which causes checks to fail.

One year of mirrorlogs keeps a table of 5000 MB which is rather large as
we only show 7 days of logs. Keep it 6 months as maybe in the future
mirror operators are interested in older data.

Keep postgresql connections around for 5 minutes this avoids expensive
authentication requests.

certbot switched to ECDSA by default about two years ago, following
[recommended practices][1].

We are currently using RSA with 4096 bits, which is extremely slow to
sign. Using ECDSA should give us a nice speedup.

[1]: https://wiki.mozilla.org/Security/Server_Side_TLS#Intermediate_compatibility_.28recommended.29

This reduces the session cache size and adds the
`DHE-RSA-CHACHA20-POLY1305` cipher.

This allows serving a stale response even to the request that triggers
an update. This should ensure all requests finish quickly.

With just `proxy_cache_use_stale updating`, the request that attempts to
update the cache waits for the response, while all other requests get to
use the stale response.

Currently archweb is badly overloaded and can take over half a minute to
respond. Pacman is not that patient and fails the download.

planet.archlinux.org redirects to https://archlinux.org/planet which
then redirects to https://archlinux.org/planet/. Skip one extra
redirect.

As we aren't the speediest at checking, this is fine.

This rate limits the endpoint which does things (ie. uwsgi). 10 requests
per second was already a lot, so 5 should be fine, realistically it can
go lower as we have a burst.