This allows serving a stale response even to the request that triggers
an update. This should ensure all requests finish quickly.

With just `proxy_cache_use_stale updating`, the request that attempts to
update the cache waits for the response, while all other requests get to
use the stale response.

Currently archweb is badly overloaded and can take over half a minute to
respond. Pacman is not that patient and fails the download.

Robin Candau authored 9 months ago and Robin Candau committed 9 months ago

> 2024/06/02 11:05:53 \[warn\] 30324#30324: the "listen ... http2" directive is deprecated, use the "http2" directive instead

Fixes #589

Jelle van der Waa authored 1 year ago and Jelle van der Waa committed 1 year ago

Nginx does not directly support cgi scripts so we rely on fcgiwrap. All
git repositories under /srv/repos are exported if they have a special
git-daemon-export-ok file in their .git directory.

The intention of "unique" in "groups['buildservers'] | sort | unique"
was to account for combining multiple groups and passing them to the
sort and unique filters. However, with only one group it looks silly.

There is a need for build servers to never build against outdated repo
databases, even with syncrepo providing a local mirror that is updated
every minute. To that effect, we adjust mirrorlist on build servers so
the first mirror is the tier0 mirror provided by gemini.

Keep the syncrepo role on build servers in order to have a local cache
of packages and avoid concurrent build jobs downloading the same files
causing them to be corrupted.

Finally, configure gemini to use its own repos (like other mirrors do).

- Set "proxy_cache_lock on" to avoid sending more than one auth request
  to archweb when a cache entry doesn't exist for the credentials.
- Set "proxy_cache_use_stale updating" to prevent nginx sending multiple
  auth requests to archweb while the cache is being refreshed. [1]

[1] http://mailman.nginx.org/pipermail/nginx/2016-January/049734.html

Use $uri instead of $request_uri in proxy_cache_key to avoid caching
based on the original URI which is different for each pacman request.

The cache keys are now in the following format:

  httpsarchlinux.org/devel/mirrorauth/Basic <credentials>

This implements authentication to our repos.archlinux.org tier 0 mirror
via archweb.

Kristian Klausen authored 4 years ago and Jelle van der Waa committed 3 years ago

A extra access_log entry was added with the following commands:
$ cd roles
$ grep -lr access_log | xargs -P 1 -n 1 sed -i '/access_log/ s/\(.*\)\( \)\(\(reduced\|main\);$\)/\1 \3\n\1.json json_\3/'

Where http does not redirect to https. These are package mirrors and
the web key directory.

Signed-off-by: Florian Pritz <bluewind@xinu.at>

Signed-off-by: Florian Pritz <bluewind@xinu.at>

Signed-off-by: Florian Pritz <bluewind@xinu.at>

This also adds location / {} blocks to make the letsencrypt include
work.

Signed-off-by: Florian Pritz <bluewind@xinu.at>

It's used by the syncrepo script that is linked in the wiki[1].

[1] https://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors



Signed-off-by: Florian Pritz <bluewind@xinu.at>

A simple redirect breaks the syncrepo script because it doesn't follow
redirects. Copy the vhost for now and deduplicate it later.

Signed-off-by: Florian Pritz <bluewind@xinu.at>

Signed-off-by: Florian Pritz <bluewind@xinu.at>