This overlaid MR changes a couple things, namely the format of the meta package archive and only updating archives when needed.
Update When Needed:
We now use the information_schema.tables AUTO_INCREMENT column to keep track of a "serial number" that can be used to tell us whether we need to update or not.
Metadata Format:
The SQL logic in this file for package metadata now exactly
reflects RPC's search logic, without searching for specific
packages.
Two command line arguments are available:
--extended | Include License, Keywords and Groups fields
When one of these arguments are passed, the script will create
a packages-meta-ext-v1.json.gz, configured via packagesmetaextfile.
Archive JSON is in the following format: line-separated package objects
enclosed in a list:
[
{...},
{...},
{...}
]
This allows users to stream the archive and process packages
line-by-line, during the local storage process.
Signed-off-by: Kevin Morris <kevr@0cost.org>