Two new archives are available:

- packages-meta-v1.json.gz
    - RPC search formatted data for all packages
    - ~2.1MB at the time of writing.
- packages-meta-ext-v1.json.gz (via --extended)
    - RPC multiinfo formatted data for all packages.
    - ~9.8MB at the time of writing.

New dependencies are required for this update:

- `python-orjson`

All archives served out by aur.archlinux.org distribute the Last-Modified
header and support the If-Modified-Since header, which should be
populated with Last-Modified's value. These should be used by clients
to avoid redownloading the archive when unnecessary.

Additionally, the new meta archives contain a format suitable for
streaming the data as the file is retrieved. It is still in JSON
format, however, users can parse package objects line by line after
the first '[' found in the file, until the last ']'; both contained
on their own lines.

Note: This commit is a documentation change and commit body.

Signed-off-by: Kevin Morris <kevr@0cost.org>

The SQL logic in this file for package metadata now exactly
reflects RPC's search logic, without searching for specific
packages.

Two command line arguments are available:

    --extended | Include License, Keywords, Groups, relations
                 and dependencies.

When --extended is passed, the script will create a
packages-meta-ext-v1.json.gz, configured via packagesmetaextfile.

Archive JSON is in the following format: line-separated package objects
enclosed in a list:

    [
    {...},
    {...},
    {...}
    ]

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Bert Peters via aur-dev authored 3 years ago and Kevin Morris committed 3 years ago

This is in addition to the current recipients. Co-maintainers should
also be made aware when their package has pending requests.

This reverts commit 986fa9ee.

This reverts commit 22b3af61.

This change now requires that PHP routes do not return HTTP 404
to be considered for the /metrics population. Additionally,
we make a small sanitization here to avoid trailing '/'
characters, unless we're on the homepage route.

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Along with this initial requests metric implementation,
we also now serve the `/metrics` route, which grabs request
metrics out of cache and renders them properly for Prometheus.

**NOTE** Metrics are only enabled when the aurweb system admin
has enabled caching by configuring `options.cache` correctly
in `$AUR_CONFIG`. Otherwise, an error is logged about no cache
being configured.

New dependencies have been added which require the use of
`composer`. See `INSTALL` for the dependency section in regards
to composer dependencies and how to install them properly for
aurweb.

Metrics are in the following forms:

    aurweb_http_requests_count(method="GET",route="/some_route")
    aurweb_api_requests_count(method="GET",route="/rpc",type="search")

This should allow us to search through the requests for specific routes
and queries.

Signed-off-by: Kevin Morris <kevr@0cost.org>

This was already in the repository in ./COPYING

This reverts commit 1b452d12.

This was missing from the project and really needs to be here.

Closes #107

Signed-off-by: Kevin Morris <kevr@0cost.org>

With FastAPI starting to come closer to a close, we've got to advertise
this deprecation so that users have some time to adjust before making
the changes. We have not specified a specific time here, but we'd like
this message to reach users of the RPC API for at least a month before
any modifications are made to the interface.

Signed-off-by: Kevin Morris <kevr@0cost.org>

With a recent curl update, it now rejects URLs with spaces in it.
We should probably fix this so that we can sanitize urls with spaces
to be used properly, but for now, just remove spaces in the directory.

Signed-off-by: Kevin Morris <kevr@0cost.org>

turret-dev authored 3 years ago and Eli Schwartz committed 3 years ago

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>

If the person running the tests has a global configuration to sign git
commits, this breaks the testsuite which looks for a key capable of
committing dummy data under a dummy author

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>

https://lists.archlinux.org/pipermail/arch-dev-public/2021-June/030462.html

Use a pure vanilla JavaScript typeahead implementation to finally
deprecate the old jQuery version and typeahead library.

All the arch-x@archlinux.org -> arch-x@lists.archlinux.org aliases will
be dropped soon[1].

[1] https://lists.archlinux.org/pipermail/arch-dev-public/2021-June/030462.html

Jelle van der Waa authored 3 years ago and Eli Schwartz committed 3 years ago

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>

Jelle van der Waa authored 3 years ago and Eli Schwartz committed 3 years ago

Use a CSS animation for jQuery.Animate and replace the rest with pure
vanilla JavaScript.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>

Jelle van der Waa authored 3 years ago and Eli Schwartz committed 3 years ago

The Document.execCommand API is deprecated and no longer recommended to
be used. It's replacement is the much simpler navigator.clipboard API
which is supported in all browsers except internet explorer.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>

Steven Guikal authored 3 years ago and Eli Schwartz committed 3 years ago

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>

Steven Guikal authored 3 years ago and Eli Schwartz committed 3 years ago

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Some of the columns that were changed still want to be
case insensitive. Good thing our tables have nice
separation.

Signed-off-by: Kevin Morris <kevr@0cost.org>

Leonidas Spyropoulos authored 3 years ago and Kevin Morris committed 3 years ago

MySql defaults to `utf8` and case insensitive collation so migrate these to case sensitive and `utf8mb4`

Closes #21

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>

Leonidas Spyropoulos authored 3 years ago and Kevin Morris committed 3 years ago

Closes: #34

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>

op.drop_constraint requires a valid field to drop the constraint on.
Without this, downgrade cannot occur.

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Signed-off-by: Kevin Morris <kevr@0cost.org>

Marcus Andersson authored 3 years ago and Eli Schwartz committed 3 years ago

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>

I've seen this happen a bunch of times now. Someone cannot push to the
AUR, and the error report is some traceback with a KeyError which is
difficult to understand without context:

remote: Traceback (most recent call last):
remote:   File "/srv/http/aurweb/aur.git/hooks/update", line 33, in <module>
remote:     sys.exit(load_entry_point('aurweb==5.0.0', 'console_scripts', 'aurweb-git-update')())
remote:   File "/usr/lib/python3.9/site-packages/aurweb-5.0.0-py3.9.egg/aurweb/git/update.py", line 306, in main
remote: KeyError: 'pkgbase'

Eventually it turns out that their .SRCINFO file is... badly corrupted.
Generally, they managed to accidentally commit an *empty* file instead
of a .SRCINFO, and in all cases, the problem was on the very first
lookup for 'pkgbase'.

Point people to the actual failing commit, and have a nicely formatted
message indicating that the .SRCINFO is completely invalid.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>