Signed-off-by: Christian Heusel <christian@heusel.eu>

This should i.e. forbid crawlers to index all of the git diffs which
put's unneccessary load on the server and is not really of benefit to be
indexed anyways.

Link: archlinux/infrastructure#610


Reviewed-by: Sven-Hendrik Haase <svenstaro@gmail.com>
Reviewed-by: Levente Polyak <anthraxx@archlinux.org>
Signed-off-by: Christian Heusel <christian@heusel.eu>

archwiki: Add simple challenge for Chinese IP addresses

See merge request !851

The wiki has been hammered with requests from some stupid Chinese
bots/crawlers. Adding a simple challenge (requiring a cookie to be set),
seems to be enough to throw them off.

This was initially added for all pages, but as that could affect Chinese
search engines (concern raised on the forum[1]), it was changed to only
affect "action views", which search engines are not supposed to crawl.

[1] https://bbs.archlinux.org/viewtopic.php?pid=2185963#p2185963

This will be used for installing the geoip2 module, so we can make it
more difficult for Chinese bots to crawl the wiki.

The name of the shared object file can be overridden in case it is not
named ngx_http_{{ module.name }}_module.so, e.g. srcache where the
shared object is named ngx_http_srcache_filter_module.so.

Archweb now exports Prometheus status via /metrics with request duration
information.

Add support for "legacy" RSA 4096 certs

Closes releng#22

See merge request archlinux/infrastructure!852

To shut up the linter.

Using a cert named after the primary domain with `_legacy` appended.
However, the cert is only issued for the legacy domains, not the primary
domain.

Deploy for `ipxe.archlinux.org`.

Fixes: archlinux/releng#22

They might conflict with the normal configuration, so we don't want
these redirects to get cached.

With RSA 4096 instead of ECDSA.

Don't require the cert to have the same name as the first domain.

This reverts commit f5b566fa.

We no longer have cloud servers with 1-vCPU so this isn't needed.

Simplify the role by removing the configurability of "EditionIDs", and
hardcoding its value to "GeoLite2-Country GeoLite2-City". While it was
originally intended for consumers to select which database(s) to fetch,
it's not straightforward how to support multiple inclusions of the role.

This reverts commit a9b596c4.

We no longer have cloud servers with 1-vCPU so this isn't needed.

dbscripts: Unbreak mirrorauth

See merge request archlinux/infrastructure!849

archlinux.org started rejecting connections without SNI because of
experiments with deploying HTTP/3.

See: archlinux/infrastructure!850

nginx: Set a resolver

Closes #607

See merge request archlinux/infrastructure!848

This is required of OCSP stapling to work, or you get warnings when
NGINX starts up:

    no resolver defined to resolve e6.o.lencr.org while requesting certificate status

Let NGINX use the local systemd-resolved as its resolver.

Fixes: #607

After fixing the cert setup on the host a working certbot installation
is needed for the automatic cert renewal.

Signed-off-by: Christian Heusel <christian@heusel.eu>

Fixes: 7fdcc769 ("Revert "dbscripts: Add tmpfiles.d/rsyncd.conf for abs and friends"")
Signed-off-by: Christian Heusel <christian@heusel.eu>

Ensure atomicity for lastsync file

See merge request !794

pita strudl authored 8 months ago and pita strudl committed 8 months ago

We try to ensure an atomic operation of the lastsync file. This requires creating a tmp file which needs to be ignored.
This should take care of having empty lastsync files being served.
Possible cause is that the IO is stuck thus taking several seconds to write the lastsync timestamp.
This causes mirrors to download the empty file which causes checks to fail.

One year of mirrorlogs keeps a table of 5000 MB which is rather large as
we only show 7 days of logs. Keep it 6 months as maybe in the future
mirror operators are interested in older data.

Keep postgresql connections around for 5 minutes this avoids expensive
authentication requests.

certbot: Use ECDSA (P-256) certificates, not RSA

See merge request !844

certbot switched to ECDSA by default about two years ago, following
[recommended practices][1].

We are currently using RSA with 4096 bits, which is extremely slow to
sign. Using ECDSA should give us a nice speedup.

[1]: https://wiki.mozilla.org/Security/Server_Side_TLS#Intermediate_compatibility_.28recommended.29

nginx: Update SSL settings to current guidelines

See merge request !843

This reduces the session cache size and adds the
`DHE-RSA-CHACHA20-POLY1305` cipher.

dbscripts: Enable proxy_cache_background_update for mirrorauth

See merge request !842

This allows serving a stale response even to the request that triggers
an update. This should ensure all requests finish quickly.

With just `proxy_cache_use_stale updating`, the request that attempts to
update the cache waits for the response, while all other requests get to
use the stale response.

Currently archweb is badly overloaded and can take over half a minute to
respond. Pacman is not that patient and fails the download.

planet.archlinux.org redirects to https://archlinux.org/planet which
then redirects to https://archlinux.org/planet/. Skip one extra
redirect.

As we aren't the speediest at checking, this is fine.

This rate limits the endpoint which does things (ie. uwsgi). 10 requests
per second was already a lot, so 5 should be fine, realistically it can
go lower as we have a burst.