This enables us to calculate the cache hit ratio, which may help
determine whether more caching would be beneficial.

Please note that this only counts requests for which caching is enabled
(e.g. {fastcgi,proxy}_cache is configured), e.g. for static served files
cache_status will be "".

[1] http://nginx.org/en/docs/http/ngx_http_upstream_module.html#var_upstream_cache_status

Mainly because we are curious. The data may also be used to decide if we
want to drop older versions of TLS.

Fixes: 8dfa7e8c ("nginx: Add plumbing for enabling HTTP/3 conditionally")

We want to roll out HTTP/3 slowly, so this adds the necessary plumbing
and makes it possible to enable it per host.

Instead of adding the conditional logic to each nginx template, the 443
listen config is moved out into a snippet which is managed by the nginx
role.

HTTP/3 uses QUIC which is built on UDP. UDP is connectionless and
therefore reuseport[1][2] must be used to ensure that UDP packets for
the same QUIC connection is directed to the same worker. reuseport can
only be enabled once, so a default_server is added to the
"inventory_hostname vhost" for SSL/QUIC (reuseport is only enabled for
the latter). ssl_reject_handshake[3] is enabled as that allows enabling
SSL/QUIC without specifying a certificate.

[1] https://nginx.org/en/docs/http/ngx_http_core_module.html#listen
[2] https://lwn.net/Articles/542629/
[3] http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_reject_handshake

Ref #606

F5/nginx has blogged about this[1] and it is also mentioned in nginx's
documentation[2]:
"There could be several add_header directives. These directives are
inherited from the previous configuration level if and only if there are
no add_header directives defined on the current level. "

The problem occurs when add_header is used in a child context like a
server{} or location{} block. It is solved by moving the HSTS header
into a snippet, which is now included before all add_header lines.

For now the HSTS header is the only global header, but in the future we
may need to add more global headers, like the Alt-Svc header[3] for
HTTP/3.

[1] https://www.f5.com/company/blog/nginx/avoiding-top-10-nginx-configuration-mistakes#directive-inheritance
[2] https://nginx.org/en/docs/http/ngx_http_headers_module.html#add_header
[3] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Alt-Svc

Fix #608

This will be used for installing the geoip2 module, so we can make it
more difficult for Chinese bots to crawl the wiki.

The name of the shared object file can be overridden in case it is not
named ngx_http_{{ module.name }}_module.so, e.g. srcache where the
shared object is named ngx_http_srcache_filter_module.so.

This brings it in line with the non-JSON "reduced" log format.

Andrea Denisse Gómez Martínez authored 3 years ago and Jelle van der Waa committed 3 years ago

Closes #317.

Kristian Klausen authored 3 years ago and Jelle van der Waa committed 3 years ago

A extra access_log entry was added with the following commands:
$ cd roles
$ grep -lr access_log | xargs -P 1 -n 1 sed -i '/access_log/ s/\(.*\)\( \)\(\(reduced\|main\);$\)/\1 \3\n\1.json json_\3/'

This is much cleaner because the nginx role does not have to set the
fastcgi_cache variable to "false" by default, which was overridden by
host_vars/apollo.archlinux.org to "wiki", but the value was still
hardcoded in the config.

At first, I was wondering that the cache "zone" name should be
generalized to improve the configuration (from the original per-host to
per-service), but that would be an overkill since the fastcgi cache is
used only for the wiki...

This allows you to add snippets for toplevel nginx configuration directives that can't go into the http level.
Use this for loading modules and such.

These are static requests for JS/CSS assets which are the topmost
request for the wiki. Caching these in nginx helps a lot to turn down
the load.

This is changed globally because it is probably fine for other services
too.

Some AUR RPC requests apparently manage to hit the 4k default limit and
if they do, they get an empty response. This is an easier fix to the
problam than  changing the maximum request length in each AUR helper.

Signed-off-by: Florian Pritz <bluewind@xinu.at>

This is mostly to resolve issues on luna where nginx is hitting the
limit, but the higher limit won't hurt other machines so I'm not putting
it in a variable for now.

Signed-off-by: Florian Pritz <bluewind@xinu.at>

Signed-off-by: Florian Pritz <bluewind@xinu.at>

For proxy/fastcgi/uwsgi blocks, logging is still set to the old format,
but for everything else (= static data) a reduced format is used that
excludes items that no longer make sense (request_time, remote_user) and
those that are personal information all the time (remote_addr, http_x_forwarded_for).

Signed-off-by: Florian Pritz <bluewind@xinu.at>

Chrome reports that our JavaScript is not compressed with gzip/brotli,
this is due to chrome receiving javascript as application type:
application/javascript.

This is the same as used on luna and as expected by the zabbix nginx
monitoring service.

Signed-off-by: Florian Pritz <bluewind@xinu.at>

Suppress nginx's stdout/stderr because it's not possible to stop nginx
from logging to stderr once it has been enabled, and it's enabled by
default (via compilation option).

Move the error_log to the root context as it applies to more than http.

The logrotate setup installed by nginx doesn't actually rotate the text
log files we've been using.

planet and sources roles still install configs that use text log files.

The settings don't hurt normal http connections if they are defined at
the top level.

Signed-off-by: Florian Pritz <bluewind@xinu.at>