We do not usually expose metrics publicly and there is no good reason
for handling aurweb differently.

Fixes: 74757d6b ("Scape aurweb metrics")

They are our HTTP/3 guinea pigs for now. HTTP/3 has been enabled on
archlinux.org since 2024-07-22, so I do not expect any issues.

$http_host is changed to $host for aurweb, as HTTP/3 uses the
":authority" pseudo-header instead of the "Host" header[1][2].

[1] https://trac.nginx.org/nginx/ticket/2281
[2] https://mailman.nginx.org/pipermail/nginx-devel/2024-January/LCIUMLKCM2EBMEMTU3KXMW74AP2C4FYZ.html

Ref #606

We want to roll out HTTP/3 slowly, so this adds the necessary plumbing
and makes it possible to enable it per host.

Instead of adding the conditional logic to each nginx template, the 443
listen config is moved out into a snippet which is managed by the nginx
role.

HTTP/3 uses QUIC which is built on UDP. UDP is connectionless and
therefore reuseport[1][2] must be used to ensure that UDP packets for
the same QUIC connection is directed to the same worker. reuseport can
only be enabled once, so a default_server is added to the
"inventory_hostname vhost" for SSL/QUIC (reuseport is only enabled for
the latter). ssl_reject_handshake[3] is enabled as that allows enabling
SSL/QUIC without specifying a certificate.

[1] https://nginx.org/en/docs/http/ngx_http_core_module.html#listen
[2] https://lwn.net/Articles/542629/
[3] http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_reject_handshake

Ref #606

F5/nginx has blogged about this[1] and it is also mentioned in nginx's
documentation[2]:
"There could be several add_header directives. These directives are
inherited from the previous configuration level if and only if there are
no add_header directives defined on the current level. "

The problem occurs when add_header is used in a child context like a
server{} or location{} block. It is solved by moving the HSTS header
into a snippet, which is now included before all add_header lines.

For now the HSTS header is the only global header, but in the future we
may need to add more global headers, like the Alt-Svc header[3] for
HTTP/3.

[1] https://www.f5.com/company/blog/nginx/avoiding-top-10-nginx-configuration-mistakes#directive-inheritance
[2] https://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header
[3] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Alt-Svc

Fix #608

This should i.e. forbid crawlers to index all of the git diffs which
put's unneccessary load on the server and is not really of benefit to be
indexed anyways.

Link: #610


Reviewed-by: Sven-Hendrik Haase <svenstaro@gmail.com>
Reviewed-by: Levente Polyak <anthraxx@archlinux.org>
Signed-off-by: Christian Heusel <christian@heusel.eu>

Robin Candau authored 9 months ago and Robin Candau committed 9 months ago

> 2024/06/02 11:05:53 \[warn\] 30324#30324: the "listen ... http2" directive is deprecated, use the "http2" directive instead

Fixes #589

Closes: #542
Fixes: 722cc5bf ("aurweb: release 6.2.8")

* bump version
* services: rename tuvotereminder to votereminder
* nginx: redirect /tu to /package-maintainer
* nginx: remove /trusted-user/TUbylaws.html redirect

Signed-off-by: moson <moson@archlinux.org>

Playbook allows us to provision an aurweb sandbox host.

Ref: aurweb/!752

Signed-off-by: moson <moson@archlinux.org>

Jelle van der Waa authored 1 year ago and Leonidas Spyropoulos committed 1 year ago

Apply the same rate limitting and fail2ban rules for aur.archlinux.org

Signed-off-by: Leonidas Spyropoulos <artafinde@archlinux.org>

This release removes the php code and adjusts the location of .gz
artifacts.

Signed-off-by: Leonidas Spyropoulos <artafinde@archlinux.org>

Signed-off-by: Leonidas Spyropoulos <artafinde@archlinux.org>
Co-authored-by: Kristian Klausen <kristian@klausen.dk>
Co-authored-by: moson-mo <mo-son@mailbox.org>

Expose aurweb RPC using goaurrpc to reduce the load on the server.
Additionally we can now geo-serve this ro reduce load and bandwidth.

Signed-off-by: Leonidas Spyropoulos <artafinde@archlinux.org>

Kevin Morris authored 2 years ago and Leonidas Spyropoulos committed 2 years ago

This commit brings in four new routes to nginx:
- /archives/metadata.git
- /archives/users.git
- /archives/pkgbases.git
- /archives/pkgnames.git

See https://gitlab.archlinux.org/archlinux/aurweb/-/blob/master/doc/git-archive.md



For now, we will be updating the repositories once every 10 minutes.

Signed-off-by: Leonidas Spyropoulos <artafinde@archlinux.org>
Co-signed by:  Kevin Morris <kevr@0cost.org>

The burst size of 300 reportedly allows ~150 git operations. This might
not always be sufficient when installing a lot of packages from the AUR.

Specify a higher burst size to cover most legit use cases, even if this
makes us more susceptible to abuse.

Some users scrape our git endpoint with quite some requests per second
(32) this is not something cgit/smartgit can handle and has caused the
AUR to go down once (http 502).

Jelle van der Waa authored 3 years ago and Jelle van der Waa committed 3 years ago

Make nginx serve static assets to offload gunicorn as for example
loading the home page is making 7 static requests out of 8 requests in
total. Set caching headers for now for 7 days, so browsers don't request
ideally this would be 30 days but let's keep it 7 days for now.

Kevin Morris authored 3 years ago and Kristian Klausen committed 3 years ago

Signed-off-by: Kevin Morris <kevr@0cost.org>

The latter can cause duplicate Content-Type headers. Thanks to strcat
for notifying me of the issue.

Leonidas Spyropoulos authored 3 years ago and Jelle van der Waa committed 3 years ago

Closes: #318

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>

The TU-Bylaws page is now deployed as gitlab page, making all of this
unrequired, the permanent redirect can stay for a while but the wiki is
already updated.

Leonidas Spyropoulos authored 3 years ago and Jelle van der Waa committed 3 years ago

Stop uncontrolled requests before reach php backend

Closes: #276

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>

Closes: #278

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>

Kristian Klausen authored 4 years ago and Jelle van der Waa committed 3 years ago

A extra access_log entry was added with the following commands:
$ cd roles
$ grep -lr access_log | xargs -P 1 -n 1 sed -i '/access_log/ s/\(.*\)\( \)\(\(reduced\|main\);$\)/\1 \3\n\1.json json_\3/'

Added the uwsgi_modifier1 option to nginx as described on [0] and also
change the chmod option on the socket to allow nginx to connect to it.

[0] https://gist.github.com/janoliver/85b682227bd9fcb8942885e60208bd76

Added a smartgit_socket option to the defaults. Reworked the tasks package
installation to look cleaner and also separated the cgit and git package installations
so we can trigger uwsgi reloads on updates. Changed the tubylaws repo update variable
to trigger the bylaws changes only when the tubylaws repository change, not the aurweb
one. Added tasks to install the apcu configuration, cgit uwsgi ini file, cgit rc file and
smartgit uwsgi ini file. Trigger an uwsgi reload in case the cgit-aurweb or git packages
change. Also added a few missing options to the aurweb configuration file. Rework the nginx
configuration file to use the cgit and smartgit uwsgi services.

To make things consistent, rename the role to aurweb.

Jelle van der Waa authored 7 years ago and Giancarlo Razzolini committed 4 years ago

The ansible role for the Arch User Repository.

Thanks-to: Eli Schwartz <eschwartz@archlinux.org>