1. 06 Jun, 2021 40 commits
    • Kevin Morris's avatar
      add case [in]sensitivity tests + add OfficialProvider model · 1874e821
      Kevin Morris authored
      
      
      `ci` in this context means "Case Insensitive".
      `cs` in this context means "Case Sensitive".
      
      New models created:
          - OfficialProvider
            This was required to write a test for checking that
            OfficialProviders behaves as we expect, which was the starter
            for the original aurblup bug.
      
      New tests created:
          - test_official_provider
      
      Modified tests:
          - test_package_base: add ci test
          - test_package: add ci test
          - test_session: add cs test
          - test_ssh_pub_key: add cs test
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      1874e821
    • Kevin Morris's avatar
      remove unused imports, rectify isort violations · 5ceeb88b
      Kevin Morris authored
      
      
      Files got into the branch that violate both PEP-8 guidelines
      and isorts. This fixes them.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      5ceeb88b
    • Kevin Morris's avatar
      fix test_accounts_routes test coverage · 62e58b12
      Kevin Morris authored
      
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      62e58b12
    • Kevin Morris's avatar
      fix aurweb.auth test coverage · 228bc8fe
      Kevin Morris authored
      
      
      With mysqlclient, we no longer need to account for a user not existing
      when an ssh key is found.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      228bc8fe
    • Kevin Morris's avatar
      use mysql backend in config.dev · aecb6494
      Kevin Morris authored
      
      
      First off: This commit changes the default development database
      backend to mysql. sqlite, however, is still completely supported
      with the caveat that a user must now modify config.dev to use
      the sqlite backend.
      
      While looking into this, it was discovered that our SQLAlchemy
      backend for mysql (mysql-connector) completely broke model
      attributes when we switched to utf8mb4_bin (binary) -- it does
      not correct the correct conversion to and from binary utf8mb4.
      
      The new, replacement dependency mysqlclient does. mysqlclient
      is also recommended in SQLAlchemy documentation as the "best"
      one available.
      
      The mysqlclient backend uses a different exception flow then
      sqlite, and so tests expecting IntegrityError has to be modified
      to expect OperationalError from sqlalchemy.exc.
      
      So, for each model that we define, check keys that can't be
      NULL and raise sqlalchemy.exc.IntegrityError if we have to.
      This way we keep our exceptions uniform.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      aecb6494
    • Kevin Morris's avatar
      modify schema primary keys to be nullable+defaulted · d7481b96
      Kevin Morris authored
      
      
      This fixes SQLAlchemy warnings related to primary keys not
      having an auto_increment or nullable.
      
      We've done this by making all foreign primary keys nullable.
      
      In ApiRateLimit's case, we can set a default str to act as
      a null, which seems a bit more sensible.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      d7481b96
    • Kevin Morris's avatar
      add ApiRateLimit SQLAlchemy ORM model · a65a6060
      Kevin Morris authored
      
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      a65a6060
    • Kevin Morris's avatar
      add PackageRelation SQLAlchemy ORM model · 2b83d2fb
      Kevin Morris authored
      
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      2b83d2fb
    • Kevin Morris's avatar
      add RelationType SQLAlchemy ORM model · a9cfbce1
      Kevin Morris authored
      
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      a9cfbce1
    • Kevin Morris's avatar
      e401b92a
    • Kevin Morris's avatar
      add DependencyType SQLAlchemy ORM model · 068c8ba6
      Kevin Morris authored
      
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      068c8ba6
    • Kevin Morris's avatar
      add PackageGroup SQLAlchemy ORM model · 4201348d
      Kevin Morris authored
      
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      4201348d
    • Kevin Morris's avatar
      add PackageLicense SQLAlchemy ORM model · 75cc0be1
      Kevin Morris authored
      
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      75cc0be1
    • Kevin Morris's avatar
      add License SQLAlchemy ORM model · 943d97ef
      Kevin Morris authored
      
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      943d97ef
    • Kevin Morris's avatar
      Sanitize and modernize pytests · 38dc2bb9
      Kevin Morris authored
      
      
      Some of these tests were written before some of our convenient
      tooling existed. Additionally, some of the tests were not
      cooperating with PEP-8 guidelines or isorted.
      
      This commit does the following:
          - Replaces all calls to make_(user|session) with
            aurweb.db.create(Model, ...).
          - Replace calls to session.add(...) + session.commit() with
            aurweb.db.create.
          - Removes the majority of calls to (session|aurweb.db).delete(...).
          - Replaces session.query calls with aurweb.db.query.
          - Initializes all mutable globals in pytest fixture setup().
          - Makes mutable global declarations more concise:
            `var1, var2 = None, None` -> `var1 = var2 = None`
          - Defines a warning exclusion for test/test_ssh_pub_key.py.
          - Removes the aurweb.testing.models module.
          - Removes some useless pytest.fixture yielding.
      
      As of this commit, developers should use the following guidelines
      when writing tests:
          - Always use aurweb.db.(create|delete|query) for database
            operations, where possible.
          - Always define mutable globals in the style: `var1 = var2 = None`.
          - `yield` the most dependent model in pytest setup fixture **iff**
            you must delete records after test runs to maintain database
            integrity. Example: test/test_account_type.py.
      
      This all makes the test code look and behave much cleaner.
      Previously, aurweb.testing.setup_test_db was buggy and leaving
      objects around in SQLAlchemy's IdentityMap.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      38dc2bb9
    • Kevin Morris's avatar
      simplify test_package_keyword.py · f2121fb8
      Kevin Morris authored
      
      
      We no longer need to delete records like this; in fact, it causes
      errors now. Fix this by removing the deletions and allow
      setup_test_db to do it's job.
      
      We'll need to do this for other tests as well.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      f2121fb8
    • Kevin Morris's avatar
      add Package SQLAlchemy ORM model · 15b13326
      Kevin Morris authored
      
      
      Additionally, add an optional **kwargs passing via make_relationship.
      This allows us to use things like `uselist=False`, which was needed
      for test/test_package.py.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      15b13326
    • Kevin Morris's avatar
      add Group SQLAlchemy ORM model · b692b11f
      Kevin Morris authored
      
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      b692b11f
    • Kevin Morris's avatar
      Fix database initialization in test_term.py · e1ab02c2
      Kevin Morris authored
      
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      e1ab02c2
    • Kevin Morris's avatar
      add AcceptedTerm SQLAlchemy ORM model · 718fa48a
      Kevin Morris authored
      
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      718fa48a
    • Kevin Morris's avatar
      add Term SQLAlchemy ORM model · 29db2ee5
      Kevin Morris authored
      
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      29db2ee5
    • Kevin Morris's avatar
      add PackageKeyword SQLAlchemy ORM model · fb210158
      Kevin Morris authored
      
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      fb210158
    • Kevin Morris's avatar
      add PackageBase SQLAlchemy ORM model · a7e54981
      Kevin Morris authored
      
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      a7e54981
    • Kevin Morris's avatar
      bugfix: relax `next` verification · 822905be
      Kevin Morris authored
      
      
      AUR renders its own 404 Not Found page when a bad route
      is encountered. Introducing the previous verification
      caused an error in this case when setting a language
      while viewing the Not Found page. So, instead of checking
      through routes, just make sure that the next parameter
      starts with a '/' character, which removes the possibility
      of any cross attacks.
      
      + Removed aurweb.asgi.routes; no longer needed.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      822905be
    • Kevin Morris's avatar
      add account (view) route · 4f928b45
      Kevin Morris authored
      
      
      + Added get /account/{username} route.
      + Added account/show.html template which shows a single use
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      4f928b45
    • Kevin Morris's avatar
      add account edit (settings) routes · 4e9ef6fb
      Kevin Morris authored
      
      
      * Added account_url filter to jinja2 environment. This produces a path
        to the user's account url (/account/{username}).
      * Updated archdev-navbar to link to new edit route.
      + Added migrate_cookies(request, response) to aurweb.util, a function
        that simply migrates the request cookies to response and returns it.
      + Added account_edit tests to test_accounts_routes.py.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      4e9ef6fb
    • Kevin Morris's avatar
      add user registration routes · c94793b0
      Kevin Morris authored
      
      
      * Added /register get and post routes.
      + Added default attributes to AnonymousUser, including a new
        AnonymousList which behaves like an sqlalchemy relationship
        list.
      + aurweb.util: Added validation functions for various user fields
        used throughout registration.
      + test_accounts_routes: Added get|post register route tests.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      c94793b0
    • Kevin Morris's avatar
      add openssh to test dependencies · 19b4a896
      Kevin Morris authored
      
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      19b4a896
    • Kevin Morris's avatar
      add aurweb.captcha, a CAPTCHA utility module · df0a637d
      Kevin Morris authored
      
      
      This CAPTCHA workflow is the same workflow used by our current
      PHP implementation of account registration.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      df0a637d
    • Kevin Morris's avatar
      add aurweb.time module · 9052688e
      Kevin Morris authored
      
      
      This module includes timezone-based utilities for a FastAPI request.
      This commit introduces use of the AURTZ cookie within get_request_timezone.
      This cookie should be set to the user or session's timezone.
      
      * `make_context` has been modified to parse the request's timezone
        and include the "timezone" and "timezones" variables, along with
        a timezone specified "now" date.
      + Added `Timezone` attribute to aurweb.testing.requests.Request.user.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      9052688e
    • Kevin Morris's avatar
      aurweb.auth: add user credentials and matcher functions · 07d5907e
      Kevin Morris authored
      
      
      This clones the behavior already present in the PHP implementation,
      but it uses a global dict with credential constant keys to
      validation functions to determine if a given user has a credential.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      07d5907e
    • Kevin Morris's avatar
      add SSHPubKey ORM model · 670f711b
      Kevin Morris authored
      
      
      Includes `aurweb.models.ssh_pub_key.get_fingerprint(pubkey)` helper.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      670f711b
    • Kevin Morris's avatar
      add authenticated User LangPreference tracking · 9fdbe3f7
      Kevin Morris authored
      
      
      + Use User.LangPreference when there is no set AURSID
        if request.user.is_authenticated is true.
      + Updated post /language to update LangPreference when
        request.user.is_authenticated.
      + Restore language during test where we change it.
      + Added the user attribute to aurweb.testing.requests.Request.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      9fdbe3f7
    • Kevin Morris's avatar
      add passreset routes · a33d076d
      Kevin Morris authored
      
      
      Introduced `get|post` `/passreset` routes. These routes mimic the
      behavior of the existing PHP implementation, with the exception of
      HTTP status code returns.
      
      Routes added:
          GET /passreset
          POST /passreset
      
      Routers added:
          aurweb.routers.accounts
      
      * On an unknown user or mismatched resetkey (where resetkey must ==
        user.resetkey), return HTTP status NOT_FOUND (404).
      * On another error in the request, return HTTP status BAD_REQUEST (400).
      
      Both `get|post` routes requires that the current user is **not**
      authenticated, hence `@auth_required(False, redirect="/")`.
      
      + Added auth_required decorator to aurweb.auth.
      + Added some more utility to aurweb.models.user.User.
      + Added `partials/error.html` template.
      + Added `passreset.html` template.
      + Added aurweb.db.ConnectionExecutor functor for paramstyle logic.
        Decoupling the executor logic from the database connection logic
        is needed for us to easily use the same logic with a fastapi
        database session, when we need to use aurweb.scripts modules.
      
      At this point, notification configuration is now required to complete
      tests involved with notifications properly, like passreset.
      `conf/config.dev` has been modified to include [notifications] sendmail,
      sender and reply-to overrides. Dockerfile and .gitlab-ci.yml have been
      updated to setup /etc/hosts and start postfix before running tests.
      
      * setup.cfg: ignore E741, C901 in aurweb.routers.accounts
      
      These two warnings (shown in the commit) are not dangerous and a bi-product
      of maintaining compatibility with our current code flow.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      a33d076d
    • Kevin Morris's avatar
      implement login + logout routes and templates · 5d4a5ded
      Kevin Morris authored
      
      
      + Added route: GET `/login` via `aurweb.routers.auth.login_get`
      + Added route: POST `/login` via `aurweb.routers.auth.login_post`
      + Added route: GET `/logout` via `aurweb.routers.auth.logout`
      + Added route: POST `/logout` via `aurweb.routers.auth.logout_post`
      * Modify archdev-navbar.html template to toggle displays on auth state
      + Added login.html template
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      5d4a5ded
    • Kevin Morris's avatar
      add aurweb.auth and authentication to User · 56f27982
      Kevin Morris authored
      
      
      + Added aurweb.auth.AnonymousUser
          * An instance of this model is returned as the request user
            when the request is not authenticated
      + Added aurweb.auth.BasicAuthBackend
      + Add starlette's AuthenticationMiddleware to app middleware,
        which uses our BasicAuthBackend facility
      + Added User.is_authenticated()
      + Added User.authenticate(password)
      + Added User.login(request, password)
      + Added User.logout(request)
      + Added repr(User(...)) representation
      + Added aurweb.auth.auth_required decorator.
      
      This change uses the same AURSID logic in the PHP implementation.
      
      Additionally, introduce a few helpers for authentication,
      one of which being `User.update_password(password, rounds = 12)`
      where `rounds` is a configurable number of salt rounds.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      56f27982
    • Kevin Morris's avatar
      add aurweb.models.session.Session ORM database object · 1922e538
      Kevin Morris authored
      
      
      + Added aurweb.util module.
          - Added make_random_string function.
      + Added aurweb.db.make_random_value function.
          - Takes a model and a column and introspects them to figure out the
            proper column length to create a random string for; then creates
            a unique string for that column.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      1922e538
    • Kevin Morris's avatar
      add aurweb.models.ban.Ban ORM mapping · adc9fccb
      Kevin Morris authored
      
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      adc9fccb
    • Kevin Morris's avatar
      aurweb.db: add query, create, delete helpers · a836892c
      Kevin Morris authored
      
      
      Takes sqlalchemy kwargs or stanzas:
      
      query(Model, Model.Column == value)
      query(Model, and_(Model.Column == value, Model.Column != "BAD!"))
      
      Updated tests to reflect the new utility and a comment about upcoming
      function deprecation is added to get_account_type().
      
      From here on, phase out the use of get_account_type().
      
      + aurweb.db: Added create utility function
      + aurweb.db: Added delete utility function
      
      The `delete` function can be used to delete a record by search
      kwargs directly.
      
      Example:
          delete(User, User.ID == 6)
      
      All three functions added in this commit are typically useful to
      perform these operations without having to import aurweb.db.session.
      Removes a bit of redundancy overall.
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      a836892c
    • Kevin Morris's avatar
      move aurweb.testing to its own package · 5185df62
      Kevin Morris authored
      
      
      + Added aurweb.testing.setup_test_db(*tables)
      + Added aurweb.testing.models.make_user(**kwargs)
      + Added aurweb.testing.models.make_session(**kwargs)
      + Added aurweb.testing.requests.Client
      + Added aurweb.testing.requests.Request
      * Updated test_l10n.py to use our new Request
      Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
      5185df62