Skip to content
Snippets Groups Projects

mkpkglists improvements

Merged mkpkglists improvements
kevr/aurweb:teapot into master
All threads resolved!
Merged Kevin Morris requested to merge kevr/aurweb:teapot into master
All threads resolved!
Viewing commit cdca8bd2
Prev
Show latest version
1 file
+ 3
− 2
Compare changes
  • Side-by-side
  • Inline
  • cdca8bd2 
    Two new archives are available:

- packages-meta-v1.json.gz
    - RPC search formatted data for all packages
    - ~2.1MB at the time of writing.
- packages-meta-ext-v1.json.gz (via --extended)
    - RPC multiinfo formatted data for all packages.
    - ~9.8MB at the time of writing.

New dependencies are required for this update:

- `python-orjson`

All archives served out by aur.archlinux.org distribute the Last-Modified
header and support the If-Modified-Since header, which should be
populated with Last-Modified's value. These should be used by clients
to avoid redownloading the archive when unnecessary.

Additionally, the new meta archives contain a format suitable for
streaming the data as the file is retrieved. It is still in JSON
format, however, users can parse package objects line by line after
the first '[' found in the file, until the last ']'; both contained
on their own lines.

Note: This commit is a documentation change and commit body.

Signed-off-by: Kevin Morris's avatarKevin Morris <kevr@0cost.org>
doc/maintenance.txt
+ 3
− 2
@@ -70,7 +70,8 @@ computations and clean up the database:
* aurweb-pkgmaint automatically removes empty repositories that were created
within the last 24 hours but never populated.
* aurweb-mkpkglists generates the package list files.
* aurweb-mkpkglists generates the package list files; it takes an optional
--extended flag, which additionally produces multiinfo metadata.
* aurweb-usermaint removes the last login IP address of all users that did not
login within the past seven days.
@@ -79,7 +80,7 @@ These scripts can be installed by running `python3 setup.py install` and are
usually scheduled using Cron. The current setup is:
----
*/5 * * * * aurweb-mkpkglists
*/5 * * * * aurweb-mkpkglists [--extended]
1 */2 * * * aurweb-popupdate
2 */2 * * * aurweb-aurblup
3 */2 * * * aurweb-pkgmaint