Kevin Morris authored 2 years ago and Leonidas Spyropoulos committed 2 years ago

This commit brings in four new routes to nginx:
- /archives/metadata.git
- /archives/users.git
- /archives/pkgbases.git
- /archives/pkgnames.git

See https://gitlab.archlinux.org/archlinux/aurweb/-/blob/master/doc/git-archive.md



For now, we will be updating the repositories once every 10 minutes.

Signed-off-by: Leonidas Spyropoulos <artafinde@archlinux.org>
Co-signed by:  Kevin Morris <kevr@0cost.org>

group_vars/all was enabling just the sshd jail so move this into the
fail2ban role defaults. patchwork, security and wiki were redefining
fail2ban_jails without deviating from the group_vars/all default and
can therefore be dropped.

I have needed to use compsize on multiple occasions, and thus had to
temporarily install it. As it is already installed on 9 machines and
is generally useful, make it part of the standard tool set for btrfs.

The traffic hitting ping.archlinux.org has lately been exhausting its
default nf_conntrack_max limit of 64k. Bump it to 256k (which is also
the default limit found on systems with more than 4G of memory).

Suggested-by: Kristian Klausen <kristian@klausen.dk>

With the final lists migrated to mailman3[1], the mailman2 server can
finally be killed.

When the mailman3 server was initially setup[2], it was done on a
separate server because the mailman and mailman3 packages conflicted,
and the traffic was routed over wireguard (HTTP, LMTP and SMTP).

Instead of installing mailman3 on the original lists.al.org server and
transferring the data, it was easier just to install the missing pieces
(basically Postfix and adjusting the Nginx configuration) on the ml3
server and move the IPs (to keep the IP mail reputation).

So basically the following was done:
- The IPs for the original lists.al.org was moved to the mailman3.al.org
  server
- The mailman2 datadir was transferred to mailman3.al.org server, so we
  can keep the pipermail links alive, and import missing mails if needed
- The original lists.al.org server was decommissioned
- The mailman3.al.org server was renamed to lists.al.org
- The missing pieces was added to the mailman3 role (basically Postfix +
  Nginx adjustments)
- The mailman role was deleted and the mailman3 role renamed to mailman

[1] 75ac7d09 ("mailman: Fourth and final batch of mailman3 migrated lists")
[2] 9294828f ("Setup mailman3 server")

Fix #59

These roles are very similar and can be merged into a single new role.

Note: The archive mirror is changed from a 4-hour sync to minutely for
conformity with the other two mirrors. In practice this doesn't matter
as it was already taking over 4 hours to finish and was starting again
right after its previous run.

Ordering "when:" before "block:" makes it more readable I suppose.

Using templates anywhere but the end of the name makes grepping for
errors more difficult.

Please see [1] and [2] for a better understanding of how this works.

[1] https://doc.dovecot.org/configuration_manual/authentication/password_databases_passdb/
[2] https://doc.dovecot.org/configuration_manual/authentication/user_databases_userdb/

Fix #464

Seems ansible-lint thinks a task calling the unqualified user module is
"not valid under any of the given schemas (schema[tasks])".

Not sure why this is needed but whatever. 

https://github.com/ansible/schemas/discussions/227

The service was enabled in arch-boxes to account for "hardware clock is
not in UTC, but instead UTC+X"[1], in our case the (VM) hardware clock
is in UTC and we therfor don't need the slow systemd-time-wait-sync
service (+30 seconds).

[1] arch-boxes@e23d3c57

Fixes: 2e799bd1 ("arch_boxes_sync: Create predictable symlinks for latest image files")

arch-boxes has decided to use GitLab's package registry instead of job
artifacts[1].

[1] arch-boxes@d04c8274

Fixes: 2e799bd1 ("arch_boxes_sync: Create predictable symlinks for latest image files")

The mailing list is used for non-public communication with users, so everyone needs be able to post to it.
It is also the assigned email address of the ArchWiki user "WikiSysop".
See https://wiki.archlinux.org/title/ArchWiki:Maintenance_Team#Who,_when_and_how_to_contact

From time to time aurweb is failing with "Too many open files"
errors[1], this could indicate a bug in aurweb or perhaps the limit is
just too low. Let's try doubling the limit and see if it helps.

[1] https://gitlab.archlinux.org/archlinux/aurweb-errors/-/issues/275

The code isn't vulnerable to nginx alias traversal[1][2], nevertheless
it should only match /static/ and not e.g. /staticfoobar.

[1] d94f18a7 ("Fix nginx alias traversal")
[2] https://github.com/yandex/gixy/blob/641060d6355fbb5bd71695928a2bf14a9bcb8bf2/docs/en/plugins/aliastraversal.md

Fixes: 9294828f ("Setup mailman3 server")

Whoosh is used by default, but it is slow at indexing (multiple hours
for just aur-requests) and searching e.g. aur-requests isn't possible
(it is slow and uses 3G+ of memory resulting in it getting OOM-killed).

Xapian indexed everything in just 76 minutes and searching aur-requests
now works and is plenty fast.

Co-authored-by: Evangelos Foutras <evangelos@foutrelis.com>

This avoids triggering a GitLab push rule which rejects files that look
like secrets.

Going to be served by all our Geo boxes under riscv.mirror.pkgbuild.com.

Fixes: 578b7819 ("Capitalize the handler name in handler invocations")
Fixes: 26f289b7 ("Capitalize the first letter of all task names")

All lists have been migrated to mailman3[1] and mailman3 is what users
should use, so show its interface by default and not the mailman2
interface.

[1] 75ac7d09 ("mailman: Fourth and final batch of mailman3 migrated lists")

Fixes: 4d8dfb6a ("mailman: Third batch of mailman3 migrated lists")

arch-general
aur-general
aur-requests

It has been decided not to migrate the following unlisted and unused
lists:
arch-magazine
arch-notifications
arch-test
mailman

Fixes: 92586d5b ("change(aurweb): rework ansible config for 6.0.0")

Required for poetry 1.2 until #1917 is fixed

https://github.com/python-poetry/poetry/issues/1917



Signed-off-by: Leonidas Spyropoulos <artafinde@archlinux.org>

Signed-off-by: Leonidas Spyropoulos <artafinde@archlinux.org>