Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Gitlab 14.10 introduced a coverage_report key which obsoletes the old
way of reporting coverage data.

- directories to be linted can be configured in .gitlab-ci.yml's
  `REQUIRES_LINT` variable
- removed linting from the test job

Signed-off-by: Kevin Morris <kevr@0cost.org>

This variable allows users to override the logging.conf used
for Python logging configuration. By default, this is set
to logging.conf, which is a production config. LOG_CONFIG
is treated relative to [options] aurwebdir.

This patch allows us to specify the logging config as opposed
to copying over logging.conf in our test docker and gitlab
test scripts, as well as ease-of-testing as a developer.

Signed-off-by: Kevin Morris <kevr@0cost.org>

No longer needed; logging.conf, which is the default config
used, is now setup for production INFO logging.

Signed-off-by: Kevin Morris <kevr@0cost.org>

We'll override logging.conf with logging.test.conf for debug logging
needed for tests now, so we can rely on the default logging.conf
for production use.

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

We'll need to update the nginx config to do this; putting
this off into an MR.

This reverts commit 19bd3766.

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

No longer do we need to create any database in .gitlab-ci.

Signed-off-by: Kevin Morris <kevr@0cost.org>

The SQL logic in this file for package metadata now exactly
reflects RPC's search logic, without searching for specific
packages.

Two command line arguments are available:

    --extended | Include License, Keywords, Groups, relations
                 and dependencies.

When --extended is passed, the script will create a
packages-meta-ext-v1.json.gz, configured via packagesmetaextfile.

Archive JSON is in the following format: line-separated package objects
enclosed in a list:

    [
    {...},
    {...},
    {...}
    ]

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

This uses the internally defined docker volume `git_data`,
but the variable is configurable for changes in the future.

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

In addition, specify FASTAPI_BACKEND=gunicorn for deployment.

Signed-off-by: Kevin Morris <kevr@0cost.org>

Previously CGIT_CLONE_PREFIX_(PHP|FASTAPI), we found that we could
use the same env var in multiple places, including non-cgit-clone-prefix
areas.

So, they were renamed, and one additional prefix was added.

- CGIT_CLONE_PREFIX_PHP -> AURWEB_PHP_PREFIX
    - Used for cgit's clone prefix and AUR_CONFIG's aur_location for PHP
- CGIT_CLONE_PREFIX_FASTAPI -> AURWEB_FASTAPI_PREFIX
    - Used for cgit's clone prefix and AUR_CONFIG's aur_location for FastAPI
- AURWEB_SSHD_PREFIX
    - Used for aurweb's sshd clone prefix shown on package pages

Signed-off-by: Kevin Morris <kevr@0cost.org>

Kristian Klausen authored 3 years ago and Kevin Morris committed 3 years ago

The infrastructure changes are here[1].

[1] archlinux/infrastructure!492

The default recursion limit used by Docker's archlinux:base-devel
Python package becomes problematic in some cases when running tests
against our FastAPI application using starlette.testclient.TestClient
(aliased to fastapi.testclient.TestClient). starlette ends up with
test failures because it exceeds the recursion limit, but this only
happens when using the `TestClient`. When the ASGI servers are run,
this is not an issue and so in that case, the recursion limit has
not been touched.

This change uses a `TEST_RECURSION_LIMIT` environment variable to
modify the recursion limit of the FastAPI application. This variable
is, by default, only supplied when running pytests in Docker, but
can be force-supplied by the user.

TEST_RECURSION_LIMIT=10000 has been added to `.env` and `.gitlab-ci.yml`.

Signed-off-by: Kevin Morris <kevr@0cost.org>

This was required for the GitLab coverage badge to get the %
of coverage.

Signed-off-by: Kevin Morris <kevr@0cost.org>

As the new-age Python package manager, Poetry brings a lot
of good additions to the table. It allows us to more easily
deal with virtualenvs for the project and resolve dependencies.

As of this commit, `requirements.txt` is replaced by Poetry,
configured at `pyproject.toml`.

In Docker and GitLab, we currently use Poetry in a root fashion.
We should work toward purely using virtualenvs in Docker, but,
for now we'd like to move forward with other things. The project
can still be installed to a virtualenv and used on a user's system
through Poetry; it is just not yet doing so in Docker.

Modifications:

* docker/scripts/install-deps.sh
    * Remove python dependencies.
* conf/config.defaults
    * Script paths have been updated to use '/usr/bin'.
* docker/git-entrypoint.sh
    * Use '/usr/bin/aurweb-git-auth' instead of
      '/usr/local/bin/aurweb-git-auth'.

Additions:

* docker/scripts/install-python-deps.sh
    * A script used purely to install Python dependencies with Poetry.
      This has to be used within the aurweb project directory and
      requires system-wide dependencies are installed beforehand.
    * Also upgrades system-wide pip.

Signed-off-by: Kevin Morris <kevr@0cost.org>

Now, we have `docker/scripts/install-deps.sh`, a script used
by both Docker and .gitlab-ci.yml. We can now focus on changing
deps in this script along as well as documentation going forward.

Signed-off-by: Kevin Morris <kevr@0cost.org>

This also updates `test/README.md` to be a bit more specific
and precise with our current state of testing.

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Introduced `get|post` `/passreset` routes. These routes mimic the
behavior of the existing PHP implementation, with the exception of
HTTP status code returns.

Routes added:
    GET /passreset
    POST /passreset

Routers added:
    aurweb.routers.accounts

* On an unknown user or mismatched resetkey (where resetkey must ==
  user.resetkey), return HTTP status NOT_FOUND (404).
* On another error in the request, return HTTP status BAD_REQUEST (400).

Both `get|post` routes requires that the current user is **not**
authenticated, hence `@auth_required(False, redirect="/")`.

+ Added auth_required decorator to aurweb.auth.
+ Added some more utility to aurweb.models.user.User.
+ Added `partials/error.html` template.
+ Added `passreset.html` template.
+ Added aurweb.db.ConnectionExecutor functor for paramstyle logic.
  Decoupling the executor logic from the database connection logic
  is needed for us to easily use the same logic with a fastapi
  database session, when we need to use aurweb.scripts modules.

At this point, notification configuration is now required to complete
tests involved with notifications properly, like passreset.
`conf/config.dev` has been modified to include [notifications] sendmail,
sender and reply-to overrides. Dockerfile and .gitlab-ci.yml have been
updated to setup /etc/hosts and start postfix before running tests.

* setup.cfg: ignore E741, C901 in aurweb.routers.accounts

These two warnings (shown in the commit) are not dangerous and a bi-product
of maintaining compatibility with our current code flow.

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Kristian Klausen authored 3 years ago and Kevin Morris committed 3 years ago

[1] https://docs.gitlab.com/ee/user/project/merge_requests/test_coverage_visualization.html