"poetry run" is very slow[1] and adds +1 second to the startup time.
This is made even worse by the fact that aurweb-git-serve is called
twice by sshd[2].

[1] https://github.com/python-poetry/poetry/issues/3502
[2] https://security.stackexchange.com/questions/123795/authorizedkeyscommand-of-sshd-config-getting-called-twice/123801#123801

Fixes: 68ec7871 ("aurweb: Mirror aur.git to GitHub[1]")

The burst size of 300 reportedly allows ~150 git operations. This might
not always be sufficient when installing a lot of packages from the AUR.

Specify a higher burst size to cover most legit use cases, even if this
makes us more susceptible to abuse.

This is meant as a extra "backup" and as another way for our users to
fetch PKGBUILDs from the AUR. It also allows the community to create
their own (perhaps better) "AUR" API/database as all essential data is
now available (this + [2]).

At the monent this is experimental and we aren't committing to keeping
it around.

[1] https://github.com/archlinux/aur
[2] http://aur.archlinux.org/packages-meta-ext-v1.json.gz

Some users scrape our git endpoint with quite some requests per second
(32) this is not something cgit/smartgit can handle and has caused the
AUR to go down once (http 502).

Jelle van der Waa authored 3 years ago and Jelle van der Waa committed 3 years ago

Make nginx serve static assets to offload gunicorn as for example
loading the home page is making 7 static requests out of 8 requests in
total. Set caching headers for now for 7 days, so browsers don't request
ideally this would be 30 days but let's keep it 7 days for now.

Kevin Morris authored 3 years ago and Evangelos Foutras committed 3 years ago

[foutrelis: add vault variables described in !532]

Signed-off-by: Kevin Morris <kevr@0cost.org>
Signed-off-by: Evangelos Foutras <evangelos@foutrelis.com>

The two secrets: vault_aurweb_{secret,postmaster}

Kevin Morris authored 3 years ago and Kristian Klausen committed 3 years ago

Signed-off-by: Kevin Morris <kevr@0cost.org>

Kevin Morris authored 3 years ago and Kristian Klausen committed 3 years ago

To call these externally, poetry run must be used. This
patch brings in some wrappers that call `poetry run`
within `{{ aurweb_dir }}`.

Signed-off-by: Kevin Morris <kevr@0cost.org>

Kevin Morris authored 3 years ago and Kristian Klausen committed 3 years ago

Signed-off-by: Kevin Morris <kevr@0cost.org>

[1] aurweb!233

The latter can cause duplicate Content-Type headers. Thanks to strcat
for notifying me of the issue.

Leonidas Spyropoulos authored 3 years ago and Jelle van der Waa committed 3 years ago

Closes: #318

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>

Set the aurweb_location configuration so aur-dev emails have the correct
links in the registration url. Add new dependencies to the aurweb role
for the aurweb-notify script to be able to run on the pu branch.

The TU-Bylaws page is now deployed as gitlab page, making all of this
unrequired, the permanent redirect can stay for a while but the wiki is
already updated.

Leonidas Spyropoulos authored 3 years ago and Jelle van der Waa committed 3 years ago

Stop uncontrolled requests before reach php backend

Closes: #276

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>

Closes: #278

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>

Kristian Klausen authored 3 years ago and Jelle van der Waa committed 3 years ago

A extra access_log entry was added with the following commands:
$ cd roles
$ grep -lr access_log | xargs -P 1 -n 1 sed -i '/access_log/ s/\(.*\)\( \)\(\(reduced\|main\);$\)/\1 \3\n\1.json json_\3/'

This service only requires MySQL access and ability to submit an email.

This service only requires MySQL access and access to the aur home
directory (/srv/http/aurweb) to write the package/pkgbase/user files.

This service only requires MySQL access, network connection access and
the ability to create an pyalpm handle to sync the pacman db's to update
the blacklist.

This service only requires MySQL access for deleting old, empty reserved
aurweb pkgbases.

This service only requires MySQL access for updating the per-package
popularity counts.

The mirrors.kernel.org mirror seems to have network issues for us from
aur.archlinux.org which makes the aurweb-aurblup service fail from time
to time. Switch to our own mirror as we are in control.

According to 'doc/maintenance.txt' the usermaint service needs to be
every 2 hours. To remove the last login IP address of all users that did
not login for 7 days.

Closes: #106

Leonidas Spyropoulos authored 4 years ago and Morten Linderud committed 4 years ago

Fixes #68997

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>
Signed-off-by: Morten Linderud <morten@linderud.pw>

Extend the memcached service for the AUR to allow the memcached group to
read the socket to obtain statistics.

As all these services do maintenance tasks and query the database so
it's better to spread out the load.

The fingerprints for the RSA and ECDSA key were swapped, fixed them.

Added the new ssh host keys fingerprints to the config.

When installing the units, systemd was ignoring the Type of the unit
due to the capital O. Change it for the proper type.

For security reasons, switch to using a socket for memcached on aurweb.

Added RandomizedDelaySec to make the aurweb-git.service to start on
a different time than the borg backup, so the machine does not hammered
by both running at the same time.

Since we are using Include, remove the sshd_config file.

Added support for the aurweb role to the new openssh include mechanism,
that's baked into our sshd role.

Added the uwsgi_modifier1 option to nginx as described on [0] and also
change the chmod option on the socket to allow nginx to connect to it.

[0] https://gist.github.com/janoliver/85b682227bd9fcb8942885e60208bd76

Add memcached to the playbook and also change the php extensions to use memcached.
Removed the apcu options from defatuls and added memcached settings. Added the php-memcached
packages to the list of needed packages and also remove the apcu tasks. Added the memcached
systemd unit file and enable and start it from the tasks.