Skip to content

Better Docker, change test db name

Kevin Morris requested to merge kevr/aurweb:pu_test_config into pu

See commits.

Additionally made test_db more generic to account for repository-specific configs (duh).

Here's the comment for Docker, which is a nice Docker change!

    Improve Docker ecosystem

    Instead of using Dockerfile for everything, we've introduced
    a docker-compose.yml file and kept the Dockerfile to producing
    a pure base image for the services defined.

    docker-compose services:

    - `mariadb` - Setup mariadb
    - `sharness` - Run sharness suites
    - `pytest-mysql` - Run pytest suites with MariaDB
    - `pytest-sqlite` - Run pytest suites with SQLite
    - `test` - Run all tests and produce a collective coverage report
        - This target mounts a cache volume and copies any successful
          coverage report back to `./cache/.coverage`. Users can run
          `./util/fix-coverage ./cache/.coverage` to rewrite source
          code paths and move coverage into place to view reports
          on your local system.

    == Get Started ==

    Build `aurweb:latest`.

        $ docker build -t aurweb:latest .

    Run all tests via `docker-compose`.

        $ docker-compose up test

    You can also purely run `pytest` in SQLite or MariaDB modes.

        $ docker-compose up pytest-sqlite
        $ docker-compose up pytest-mysql

    Or `sharness` alone, which only uses SQLite internally.

        $ docker-compose up sharness

    After running tests, coverage reports are stored in `./cache/.coverage`.
    This database was most likely created in a different path, and so it
    needs to be sanitized with `./util/fix-coverage`.

        $ ./util/fix-coverage cache/.coverage
        Copied coverage db to /path/to/aurweb/.coverage.
        $ coverage report
        ...
        $ coverage html
        $ coverage xml
        ...

    Defined components:

    **Entrypoints**

    - mariadb-entrypoint.sh - setup mariadb and run its daemon
    - test-mysql-entrypoint.sh - setup mysql configurations
    - test-sqlite-entrypoint.sh - setup sqlite configurations
    - tests-entrypoint.sh - setup mysql and sqlite configurations

    **Scripts**

    - run-mariadb.sh - setup databases
    - run-pytests.sh - run pytest suites
    - run-sharness.sh - run sharness suites
    - run-tests.sh - run both pytests and sharness

    **Health**

    - mariadb.sh - A healthcheck script for the mariadb service
    - pytest.sh - A healthcheck script for the pytest-* services
    - sharness.sh - A healthcheck script for the sharness service

    This Docker configuration is setup for tests, but should be
    extendable for web and git servers.

    **Changes to Makefile**

    - Remove `.coverage` in the `clean` target
    - Add a `coverage` target which prints a report and outputs xml

    Signed-off-by: Kevin Morris <kevr@0cost.org>

    Docker: add [c]git, nginx, fastapi, php-fpm, ca

    Now, we have a full collection of services used to run
    aurweb over HTTPS using a self-signed CA.

    New Docker services:

    - `ca` - Certificate authority services
        - When the `ca` service is run, it will (if needed) generate
          a CA certificate and leaf certificate for localhost AUR
          access. This ca is then shared with things like nginx to
          use the leaf certificate. Users can import
          `./cache/ca.root.pem` into their browser or ca-certificates
          as a root CA who issued aurweb's certificate.
    - `git` - Start sshd and set it up for aur git access
    - `cgit` - Serve cgit with uwsgi on port 3000
    - `fastapi` - Serve our FastAPI app with `hypercorn` on port 8000
    - `php-fpm` - Serve our PHP-wise aurweb
    - `nginx` - Serve FastAPI, PHP and CGit with an HTTPS certificate.
        - PHP: https://localhost:8443
        - PHP CGit: https://localhost:8443/cgit
        - FastAPI: https://localhost:8444
        - FastAPI CGit: https://localhost:8444/cgit

    Short of it: Run the following in a shell to run PHP and FastAPI
    servers on port **8443** and **8444**, respectively.

        $ docker-compile up nginx

    Docker: add [c]git, nginx, fastapi, php-fpm, ca

    Now, we have a full collection of services used to run
    aurweb over HTTPS using a self-signed CA.

    This should be extended to support PHP in a follow-up
    commit on port :8080 and :8443.

    New Docker services:

    - `ca` - Certificate authority services
        - When the `ca` service is run, it will (if needed) generate
          a CA certificate and leaf certificate for localhost AUR
          access. This ca is then shared with things like nginx to
          use the leaf certificate. Users can import
          `./cache/ca.root.pem` into their browser or ca-certificates
          `./cache/ca.root.pem` into their browser or ca-certificates
          as a root CA who issued aurweb's certificate.
    - `git` - Start sshd and set it up for aur git access
    - `cgit` - Serve cgit with uwsgi on port 3000
    - `fastapi` - Serve our FastAPI app with `hypercorn` on port 8000
    - `php-fpm` - Serve our PHP-wise aurweb
    - `nginx` - Serve FastAPI and CGit with an HTTPS certificate.
        - This service can be reached via https://localhost:8443,
          cgit can be found at https://localhost:8443/cgit/.

    Short of it: Run the following in a shell to run PHP and FastAPI
    servers on port **8443** and **8444**, respectively.

        $ docker-compile up nginx

    This will host the PHP, FastAPI, CGit and Git ecosystems.

    Git SSH can be knocked at `aur@localhost:2222` as long as you have a
    valid public key in the aurweb database.

    Signed-off-by: Kevin Morris <kevr@0cost.org>

Edited by Kevin Morris

Merge request reports