Skip to content
Snippets Groups Projects
Verified Commit ebb33356 authored by Kevin Morris's avatar Kevin Morris
Browse files

update INSTALL: asgi-driven aurweb direction


We heavily attempt to provide easy use of poetry virtualenvs
with aurweb in this revision of the INSTALL file. Added a
section about cron jobs and updated the nginx config example
with a lot more detail and locations for other parts of
the AUR infrastructure.

Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
parent e5dfd53b
No related branches found
No related tags found
No related merge requests found
Pipeline #14869 waiting for manual action
......@@ -9,98 +9,131 @@ In particular, the cgit interface will be unusable as well as the ssh+git
interface. For a detailed description on how to setup a full aurweb server,
read the instructions below.
1) Clone the aurweb project:
1) Clone the aurweb project and install it (via `python-poetry`):
$ cd /srv/http/
$ git clone git://git.archlinux.org/aurweb.git
$ cd /srv/http/
$ git clone git://git.archlinux.org/aurweb.git
$ poetry install
2) Setup a web server with PHP and MySQL. Configure the web server to redirect
all URLs to /index.php/foo/bar/. The following block can be used with nginx:
server {
listen 80;
# https is preferred and can be done easily with LetsEncrypt
# or self-CA signing. Users can still listen over 80 for plain
# http, for which the [options] disable_http_login used to toggle
# the authentication feature.
listen 443 ssl http2;
server_name aur.local aur;
# To enable SSL proxy properly, make sure gunicorn and friends
# are supporting forwarded headers over 127.0.0.1 or any if
# the asgi server is contacted by non-localhost hosts.
ssl_certificate /etc/ssl/certs/aur.cert.pem;
ssl_certificate_key /etc/ssl/private/aur.key.pem;
# Asset root. This is used to match against gzip archives.
root /srv/http/aurweb/web/html;
index index.php;
location ~ ^/[^/]+\.php($|/) {
fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;
fastcgi_index index.php;
fastcgi_split_path_info ^(/[^/]+\.php)(/.*)$;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
include fastcgi_params;
# TU Bylaws redirect.
location = /trusted-user/TUbylaws.html {
return 301 https://tu-bylaws.aur.archlinux.org;
}
location ~ .* {
rewrite ^/(.*)$ /index.php/$1 last;
# smartgit location.
location ~ "^/([a-z0-9][a-z0-9.+_-]*?)(\.git)?/(git-(receive|upload)-pack|HEAD|info/refs|objects/(info/(http-)?alternates|packs)|[0-9a-f]{2}/[0-9a-f]{38}|pack/pack-[0-9a-f]{40}\.(pack|idx))$" {
include uwsgi_params;
uwsgi_pass smartgit;
uwsgi_modifier1 9;
uwsgi_param SCRIPT_FILENAME /usr/lib/git-core/git-http-backend;
uwsgi_param PATH_INFO /aur.git/$3;
uwsgi_param GIT_HTTP_EXPORT_ALL "";
uwsgi_param GIT_NAMESPACE $1;
uwsgi_param GIT_PROJECT_ROOT /srv/http/aurweb;
}
# cgitrc.proto should be configured and located somewhere
# of your choosing.
location ~ ^/cgit {
include uwsgi_params;
rewrite ^/cgit/([^?/]+/[^?]*)?(?:\?(.*))?$ /cgit.cgi?url=$1&$2 last;
uwsgi_modifier1 9;
uwsgi_param CGIT_CONFIG /srv/http/aurweb/conf/cgitrc.proto;
uwsgi_pass cgit;
}
}
Ensure to enable the pdo_mysql extension in php.ini.
# Static archive assets.
location ~ \.gz$ {
types { application/gzip text/plain }
default_type text/plain;
add_header Content-Encoding gzip;
expires 5m;
}
# For everything else, proxy the http request to (guni|uvi|hyper)corn.
# The ASGI server application should allow this request's IP to be
# forwarded via the headers used below.
# https://docs.gunicorn.org/en/stable/settings.html#forwarded-allow-ips
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Protocol ssl;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Ssl on;
}
}
3) Optionally copy conf/config.defaults to /etc/aurweb/. Create or copy
/etc/aurweb/config (this is expected to contain all configuration settings
if the defaults file does not exist) and adjust the configuration (pay
attention to disable_http_login, enable_maintenance and aur_location).
4) Install dependencies.
4) Install system-wide dependencies:
4a) Install system-wide dependencies:
# pacman -S git gpgme cgit curl openssh uwsgi uwsgi-plugin-cgi \
python-poetry
# pacman -S git gpgme cgit pyalpm python-srcinfo curl openssh \
uwsgi uwsgi-plugin-cgi php php-fpm
5) Create a new user:
4b) Install Python dependencies via poetry (required):
**NOTE** Users do not need to install pip or poetry dependencies system-wide.
You may take advantage of Poetry's virtualenv integration to manage
dependencies. This is merely a demonstration to show users how to without
a virtualenv. In Docker and CI, we don't yet use a virtualenv.
## Install Poetry dependencies system-wide, if not using a virtualenv.
# pacman -S python-pip
## Ensure pip is upgraded. Poetry depends on it being up to date.
# pip install --upgrade pip
# useradd -U -d /srv/http/aurweb -c 'AUR user' aur
# su - aur
## Install Poetry.
# curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
# export PATH="$HOME/.poetry/bin:${PATH}"
6a) Install Python dependencies via poetry:
## Use Poetry to install dependencies and the aurweb package.
# poetry lock # Resolve dependencies
# poetry update # Install/update dependencies
# poetry build # Build the aurweb package
# poetry install # Install the aurweb package and scripts
# Install the package and scripts as the aur user.
$ poetry install
When installing in a virtualenv, config.defaults must contain the correct
absolute paths to aurweb scripts, which requires modification.
6b) Setup Services
4c) Setup FastAPI Redis cache (optional).
aurweb utilizes the following systemd services:
- mariadb
- redis (optional, requires [options] cache 'redis')
- `examples/aurweb.service`
First, install Redis and start its service.
6c) Setup Cron
# pacman -S redis
# systemctl enable --now redis
Using [cronie](https://archlinux.org/packages/core/x86_64/cronie/):
Now that Redis is running, ensure that you configure aurweb to use
the Redis cache by setting `cache = redis` in your AUR config.
# su - aur
$ crontab -e
In `conf/config.defaults`, the `redis_address` configuration is set
to `redis://localhost`. This can be set to point to any Redis server
and will be used as long as `cache = redis`.
The following crontab file uses every script meant to be run on an
interval:
5) Create a new database and a user and import the aurweb SQL schema:
AUR_CONFIG='/etc/aurweb/config'
*/5 * * * * bash -c 'poetry run aurweb-aurblup'
*/5 * * * * bash -c 'poetry run aurweb-mkpkglists --extended'
*/5 * * * * bash -c 'poetry run aurweb-pkgmaint'
*/5 * * * * bash -c 'poetry run aurweb-usermaint'
*/5 * * * * bash -c 'poetry run aurweb-tuvotereminder'
*/5 * * * * bash -c 'poetry run aurweb-popupdate'
$ python -m aurweb.initdb
7) Create a new database and a user and import the aurweb SQL schema:
6) Create a new user:
$ poetry run python -m aurweb.initdb
# useradd -U -d /srv/http/aurweb -c 'AUR user' aur
7) Initialize the Git repository:
8) Initialize the Git repository:
# mkdir /srv/http/aurweb/aur.git/
# cd /srv/http/aurweb/aur.git/
......@@ -108,19 +141,26 @@ and will be used as long as `cache = redis`.
# git config --local transfer.hideRefs '^refs/'
# git config --local --add transfer.hideRefs '!refs/'
# git config --local --add transfer.hideRefs '!HEAD'
# ln -s /usr/local/bin/aurweb-git-update hooks/update
# chown -R aur .
Link to `aurweb-git-update` poetry wrapper provided at
`examples/aurweb-git-update.sh` which should be installed
somewhere as executable.
# ln -s /path/to/aurweb-git-update.sh hooks/update
It is recommended to read doc/git-interface.txt for more information on the
administration of the package Git repository.
8) Configure sshd(8) for the AUR. Add the following lines at the end of your
sshd_config(5) and restart the sshd. Note that OpenSSH 6.9 or newer is
needed!
9) Configure sshd(8) for the AUR. Add the following lines at the end of your
sshd_config(5) and restart the sshd.
If using a virtualenv, copy `examples/aurweb-git-auth.sh` to a location
and call it below:
Match User aur
PasswordAuthentication no
AuthorizedKeysCommand /usr/local/bin/aurweb-git-auth "%t" "%k"
AuthorizedKeysCommand /path/to/aurweb-git-auth.sh "%t" "%k"
AuthorizedKeysCommandUser aur
AcceptEnv AUR_OVERWRITE
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment