"poetry run" is very slow[1] and adds +1 second to the startup time.
This is made even worse by the fact that aurweb-git-serve is called
twice by sshd[2].

[1] https://github.com/python-poetry/poetry/issues/3502
[2] https://security.stackexchange.com/questions/123795/authorizedkeyscommand-of-sshd-config-getting-called-twice/123801#123801

Fixes: 68ec7871 ("aurweb: Mirror aur.git to GitHub[1]")

The burst size of 300 reportedly allows ~150 git operations. This might
not always be sufficient when installing a lot of packages from the AUR.

Specify a higher burst size to cover most legit use cases, even if this
makes us more susceptible to abuse.

This is meant as a extra "backup" and as another way for our users to
fetch PKGBUILDs from the AUR. It also allows the community to create
their own (perhaps better) "AUR" API/database as all essential data is
now available (this + [2]).

At the monent this is experimental and we aren't committing to keeping
it around.

[1] https://github.com/archlinux/aur
[2] http://aur.archlinux.org/packages-meta-ext-v1.json.gz

These are used to signal the start of the document in a stream of many
documents. As Ansible only supports one YAML document per file this is
unnecessary. About a third of our YAML documents already lacked these.

As reported by ansible-lint 6.2.1:

schema: [{'PYTHONPATH': '.'}] is not of type 'object' (schema[tasks])
roles/aurweb/tasks/main.yml:1

schema: [{'SHELL': '/bin/bash'}] is not of type 'object' (schema[tasks])
roles/dbscripts/tasks/main.yml:1

Kevin is MIA, so add my key, so we can do releases.

Kevin Morris authored 3 years ago and Kristian Klausen committed 2 years ago

Without this, we update while services are still running, and those
services depend on the AUR directory state. This commit installs
services much earlier, downs all of them, deploys out updates, then
starts them all again at the end.

Signed-off-by: Kevin Morris <kevr@0cost.org>

Some users scrape our git endpoint with quite some requests per second
(32) this is not something cgit/smartgit can handle and has caused the
AUR to go down once (http 502).

Jelle van der Waa authored 3 years ago and Jelle van der Waa committed 3 years ago

Make nginx serve static assets to offload gunicorn as for example
loading the home page is making 7 static requests out of 8 requests in
total. Set caching headers for now for 7 days, so browsers don't request
ideally this would be 30 days but let's keep it 7 days for now.

Kevin Morris authored 3 years ago and Evangelos Foutras committed 3 years ago

[foutrelis: add vault variables described in !532]

Signed-off-by: Kevin Morris <kevr@0cost.org>
Signed-off-by: Evangelos Foutras <evangelos@foutrelis.com>

The two secrets: vault_aurweb_{secret,postmaster}

Kevin Morris authored 3 years ago and Kristian Klausen committed 3 years ago

Signed-off-by: Kevin Morris <kevr@0cost.org>

Kevin Morris authored 3 years ago and Kristian Klausen committed 3 years ago

To call these externally, poetry run must be used. This
patch brings in some wrappers that call `poetry run`
within `{{ aurweb_dir }}`.

Signed-off-by: Kevin Morris <kevr@0cost.org>

Kevin Morris authored 3 years ago and Kristian Klausen committed 3 years ago

Signed-off-by: Kevin Morris <kevr@0cost.org>

[1] aurweb!233

The latter can cause duplicate Content-Type headers. Thanks to strcat
for notifying me of the issue.

Leonidas Spyropoulos authored 3 years ago and Jelle van der Waa committed 3 years ago

Closes: #318

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>

Set the aurweb_location configuration so aur-dev emails have the correct
links in the registration url. Add new dependencies to the aurweb role
for the aurweb-notify script to be able to run on the pu branch.

The TU-Bylaws page is now deployed as gitlab page, making all of this
unrequired, the permanent redirect can stay for a while but the wiki is
already updated.

Leonidas Spyropoulos authored 3 years ago and Jelle van der Waa committed 3 years ago

Stop uncontrolled requests before reach php backend

Closes: #276

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>

Closes: #278

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>

Kristian Klausen authored 4 years ago and Jelle van der Waa committed 3 years ago

A extra access_log entry was added with the following commands:
$ cd roles
$ grep -lr access_log | xargs -P 1 -n 1 sed -i '/access_log/ s/\(.*\)\( \)\(\(reduced\|main\);$\)/\1 \3\n\1.json json_\3/'

This service only requires MySQL access and ability to submit an email.

This service only requires MySQL access and access to the aur home
directory (/srv/http/aurweb) to write the package/pkgbase/user files.

This service only requires MySQL access, network connection access and
the ability to create an pyalpm handle to sync the pacman db's to update
the blacklist.

This service only requires MySQL access for deleting old, empty reserved
aurweb pkgbases.

This service only requires MySQL access for updating the per-package
popularity counts.

The mirrors.kernel.org mirror seems to have network issues for us from
aur.archlinux.org which makes the aurweb-aurblup service fail from time
to time. Switch to our own mirror as we are in control.

yaml: truthy value should be one of [false, true] (truthy)
yaml: wrong indentation: expected 4 but found 2 (indentation)
yaml: too few spaces before comment (comments)
yaml: missing starting space in comment (comments)
yaml: too many blank lines (1 > 0) (empty-lines)
yaml: too many spaces after colon (colons)
yaml: comment not indented like content (comments-indentation)
yaml: no new line character at the end of file (new-line-at-end-of-file)
load-failure: Failed to load or parse file
parser-error: couldn't resolve module/action 'hosts'. This often indicates a misspelling, missing collection, or incorrect module path.

According to 'doc/maintenance.txt' the usermaint service needs to be
every 2 hours. To remove the last login IP address of all users that did
not login for 7 days.

Closes: #106

With creates: once the aur python module is initially deployed it and
it's console-scripts will never be updated. To update the
console-scripts when the git repository is updated or when a major
Python update is done. As figuring out if a major Python update is done
we simply check if the python module is importable.

Fixes: #164

Leonidas Spyropoulos authored 4 years ago and Morten Linderud committed 4 years ago

Fixes #68997

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>
Signed-off-by: Morten Linderud <morten@linderud.pw>