They are our HTTP/3 guinea pigs for now. HTTP/3 has been enabled on
archlinux.org since 2024-07-22, so I do not expect any issues.

$http_host is changed to $host for aurweb, as HTTP/3 uses the
":authority" pseudo-header instead of the "Host" header[1][2].

[1] https://trac.nginx.org/nginx/ticket/2281
[2] https://mailman.nginx.org/pipermail/nginx-devel/2024-January/LCIUMLKCM2EBMEMTU3KXMW74AP2C4FYZ.html

Ref #606

We want to roll out HTTP/3 slowly, so this adds the necessary plumbing
and makes it possible to enable it per host.

Instead of adding the conditional logic to each nginx template, the 443
listen config is moved out into a snippet which is managed by the nginx
role.

HTTP/3 uses QUIC which is built on UDP. UDP is connectionless and
therefore reuseport[1][2] must be used to ensure that UDP packets for
the same QUIC connection is directed to the same worker. reuseport can
only be enabled once, so a default_server is added to the
"inventory_hostname vhost" for SSL/QUIC (reuseport is only enabled for
the latter). ssl_reject_handshake[3] is enabled as that allows enabling
SSL/QUIC without specifying a certificate.

[1] https://nginx.org/en/docs/http/ngx_http_core_module.html#listen
[2] https://lwn.net/Articles/542629/
[3] http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_reject_handshake

Ref #606

F5/nginx has blogged about this[1] and it is also mentioned in nginx's
documentation[2]:
"There could be several add_header directives. These directives are
inherited from the previous configuration level if and only if there are
no add_header directives defined on the current level. "

The problem occurs when add_header is used in a child context like a
server{} or location{} block. It is solved by moving the HSTS header
into a snippet, which is now included before all add_header lines.

For now the HSTS header is the only global header, but in the future we
may need to add more global headers, like the Alt-Svc header[3] for
HTTP/3.

[1] https://www.f5.com/company/blog/nginx/avoiding-top-10-nginx-configuration-mistakes#directive-inheritance
[2] https://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header
[3] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Alt-Svc

Fix #608

Signed-off-by: Leonidas Spyropoulos <artafinde@archlinux.org>

Gitlab recommends a default value of 1/4th of the total memory here, but
since the previous value was 1MB we go for a bit more conservative
approach here.

Fixes: #615


Signed-off-by: Christian Heusel <christian@heusel.eu>

In order to use tests like `body :content "text/plain" :contains "Reassigned Issue"`.

This should not cause any issues as MediaWiki should purge stale content
from the cache (by sending a PURGE request to the nginx-cache-purge
service).

We have used MediaWiki's file cache[2] until now, but recently the wiki
has been hammered with requests from some stupid Chinese bots/crawlers.

Caching at the web server level is faster as we avoid the PHP overhead
and it seems to make a difference (performance wise), especially when
the bots/crawlers are hitting us.

This is usual done with Varnish[3], but I went with a simple Python
service (30 LOC) for handling the PURGE requests as that is much simpler
thn adding Varnish to our stack.

[1] https://www.mediawiki.org/w/index.php?title=Manual:Performance_tuning&oldid=6670283#Page_view_caching
[2] https://www.mediawiki.org/wiki/Manual:File_cache
[3] https://www.mediawiki.org/wiki/Manual:Varnish_caching

Fix #315

This will be used for issue-bot[1][2].

[1] signstar#20 (comment 201743)
[2] https://gitlab.com/gitlab-org/distribution/issue-bot

See: https://github.com/element-hq/synapse/issues/17473

The need for UEFI booting originates from dedicated server and it does
not benefit cloud servers. It therefore makes sense to skip it on them.

The module postgresql_privs deprected the "password" parameter in favour
of the "login_password" parameter, therefore replace accordingly.

https://github.com/ansible-collections/community.postgresql/blob/main/CHANGELOG.rst#id19

Fixes #603



Signed-off-by: Christian Heusel <christian@heusel.eu>

This will be our backend for a Grafana-based APM.
It is sorely required for gaining insights into why application such as aurweb are slow.

We currently only enable the OTLP receiver as it seems to be the most
modern and best supported one.

We connect directly to the prometheus at localhost for the generated
metrics.

We're also using just storing traces locally in files instead of
something like S3.

sudo 1.9.15.p5-2 enables secure_path by default.

Signed-off-by: Christian Heusel <christian@heusel.eu>

This should i.e. forbid crawlers to index all of the git diffs which
put's unneccessary load on the server and is not really of benefit to be
indexed anyways.

Link: archlinux/infrastructure#610


Reviewed-by: Sven-Hendrik Haase <svenstaro@gmail.com>
Reviewed-by: Levente Polyak <anthraxx@archlinux.org>
Signed-off-by: Christian Heusel <christian@heusel.eu>

The wiki has been hammered with requests from some stupid Chinese
bots/crawlers. Adding a simple challenge (requiring a cookie to be set),
seems to be enough to throw them off.

This was initially added for all pages, but as that could affect Chinese
search engines (concern raised on the forum[1]), it was changed to only
affect "action views", which search engines are not supposed to crawl.

[1] https://bbs.archlinux.org/viewtopic.php?pid=2185963#p2185963

This will be used for installing the geoip2 module, so we can make it
more difficult for Chinese bots to crawl the wiki.

The name of the shared object file can be overridden in case it is not
named ngx_http_{{ module.name }}_module.so, e.g. srcache where the
shared object is named ngx_http_srcache_filter_module.so.

Archweb now exports Prometheus status via /metrics with request duration
information.

To shut up the linter.

Using a cert named after the primary domain with `_legacy` appended.
However, the cert is only issued for the legacy domains, not the primary
domain.

Deploy for `ipxe.archlinux.org`.

Fixes: archlinux/releng#22

They might conflict with the normal configuration, so we don't want
these redirects to get cached.

With RSA 4096 instead of ECDSA.

Don't require the cert to have the same name as the first domain.

This reverts commit f5b566fa.

We no longer have cloud servers with 1-vCPU so this isn't needed.

Simplify the role by removing the configurability of "EditionIDs", and
hardcoding its value to "GeoLite2-Country GeoLite2-City". While it was
originally intended for consumers to select which database(s) to fetch,
it's not straightforward how to support multiple inclusions of the role.

This reverts commit a9b596c4.

We no longer have cloud servers with 1-vCPU so this isn't needed.

archlinux.org started rejecting connections without SNI because of
experiments with deploying HTTP/3.

See: !850

This is required of OCSP stapling to work, or you get warnings when
NGINX starts up:

    no resolver defined to resolve e6.o.lencr.org while requesting certificate status

Let NGINX use the local systemd-resolved as its resolver.

Fixes: #607

Fixes: 7fdcc769 ("Revert "dbscripts: Add tmpfiles.d/rsyncd.conf for abs and friends"")
Signed-off-by: Christian Heusel <christian@heusel.eu>

pita strudl authored 8 months ago and pita strudl committed 8 months ago

We try to ensure an atomic operation of the lastsync file. This requires creating a tmp file which needs to be ignored.
This should take care of having empty lastsync files being served.
Possible cause is that the IO is stuck thus taking several seconds to write the lastsync timestamp.
This causes mirrors to download the empty file which causes checks to fail.

One year of mirrorlogs keeps a table of 5000 MB which is rather large as
we only show 7 days of logs. Keep it 6 months as maybe in the future
mirror operators are interested in older data.