repo_managament/*:
Remove the obsolete
`convert._transform_package_desc_to_output_package()` (the functionality
is now covered by pydantic models directly).
Change all relevant methods to be async.
Change file open commands to make use of aiofiles.

tests/*:
Remove tests for the obsolete
`convert._transform_package_desc_to_output_package()`.
Change all tests for async methods to test using pytest-asyncio.

poetry.lock:
Update lock for aiofiles and pytest-asyncio.

pyproject.toml:
Add aiofiles to the project's dependencies.
Add pytest-asyncio to the project's development dependencies.

Add tests to include instead of packages

Closes #11

See merge request archlinux/arch-repo-management!7

pyproject.toml:
Include the tests using poetry's integration for includes instead of
packages, as the latter leads to the tests being added to packages in
the generated setup.py.

Fixes #11

Add documentation and validators to models

Closes #6

See merge request archlinux/arch-repo-management!6

repo_management/models.py:
Add validators to the `BuildDate`, `CSize`, `ISize`, `Name` and
`Version` models to ensure positive integers, correct names and
versions.
Add the comparator instance methods `Version.is_older_than()` and
`Version.is_newer_than()` which rely on pyalpm's `vercmp()` to compare
two version strings and return whether they are older or newer
(respectively).
Update the documentation for `ISize` to reflect, that it represents size
of an installed package.

tests/test_models.py:
Add tests for added validators and comparator methods.

tests/test_{convert,files}.py:
Ensure that instantiation of models using the `License` model are done
correctly.

repo_management/models.py:
Document all models and sort the single attribute models alphabetically
for easier readibility.
Change the `License` model to describe a required attribute instead of
an optional (all packages need a license).
Remove the unused model `PackageFiles`.

Add integration tests for db2json and json2db

Closes #9

See merge request archlinux/arch-repo-management!5

.gitlab-ci.yml:
Rename "lint" stage to "test" stage.
Change basic target to also update the pacman files databases, as they
are required for integration tests.
Add integration target to run `tox -e integration`.

tox.ini:
Add test target that only runs tests marked "integration".

repo_management/cli.py:
Change `db2json` and `json2db` to try/except
`errors.RepoManagementError` and `argpase.ArgumentTypeError` and exit
with a non-zero return code in that case.

tests/test_cli.py:
Change the tests for `db2json` and `json2db` to cover potentially
raising `errors.RepoManagementError` and `argpase.ArgumentTypeError` and
checking whether exiting with a non-zero return code in that case.
Add parametrized integration test `test_transform_databases()` that
transforms existing pacman .files sync databases to JSON files and back
to .files repository databases again.
Add parametrized integration
`test_transform_databases_and_use_with_pacman()` to transform existing
pacman .files sync databases to JSON files and create .files and .db
repository databases from them. Afterwards pacman uses the created
repository databases to search for packages with the "linux" keyword in
them and list the files owned by the "linux" package using the created
.files sync databases.

pyproject.toml:
Add pytest configuration to setup the custom marker "integration".

repo_management/convert.py:
Change the docuemntation of `_desc_data_line_to_dicts()` to reflect,
that it may raise ValueError under certain conditions.
Change `_desc_data_to_model()` to try/except ValueError raised by
`_desc_data_line_to_dicts()` and raise a
`errors.REpomanagementValidationError` instead.

tests/test_convert.py:
Change tests expecting to raise `ValueError` to expect
`errors.RepoManagementValidationError` instead.

repo_management/commands.py:
Add `run_command()` which allows for running an external command with
the help of subprocess-tee.
Add the private method `_print_env()` which is used to print the
optional environment variables given to `run_command()`.

tests/test_commands.py:
Add tests for `_print_env()` and `run_command()`.

poetry.lock:
Update lock file for subprocess-tee 0.2.0.

pyproject.toml:
Add subprocess-tee to the list of dependencies for the project.

Add functionality to write a database file from a set of JSON files

Closes #4

See merge request archlinux/arch-repo-management!4

pyproject.toml:
Add script entrypoint for `json2db`.

repo_management/cli.py:
Add `json2db()` as entrypoint for the `json2db` script.

tests/test_cli.py:
Add a test for `json2db()`.

repo_management/argparse.py:
Add `ArgParseFactory.json2db()` as an ArgumentParser for the `json2db`
script.
Add `ArgParseFactory.string_to_writable_file_path()` for returning a
Path object, that is guaranteed to be a writable file.

tests/test_argparse.py:
Add tests for
`ArgParseFactory.{json2db,string_to_writable_file_path}()`.

repo_management/operations.py:
Add `create_db_from_json_files()` to create a repository database from
JSON files in a directory.
Change `db_file_as_models()` to rely on
`models.PackageDesc.get_output_packages()` instead of
`convert._transform_package_desc_to_output_package()` and add a missing
parameter (base) to the instantiation of `models.OutputPackageBase()`.

tests/test_operations.py:
Add the two fixtures `dummy_json_files_in_dir()` and `empty_file()` to
create a few (valid) dummy input JSON files in a temporary directory and
an empty file (respectively).
Add a test for `create_db_from_json_files()`.

repo_management/files.py:
Add `_json_files_in_directory()` yield a list of files with a .json
suffix in a directory.
Add `_read_pkgbase_json_file()` to read a JSON file representing a
pkgbase and return an `OutputPackageBase` model.
Add `_write_db_file()` to open a tar file for writing.
Add `_stream_package_base_to_db()` to stream a `models.OutputPackage()`
to a tar file (which represents a .db or a .files repository database).

tests/test_files.py:
Add tests for `_json_files_in_directory()`, `_read_pkgbase_json_file()`,
`_write_db_file()` and `_stream_package_base_to_db()`.

tests/fixtures.py:
Add `create_empty_json_files()` which creates a set of empty .json files
in a temporary directory and can be used by fixtures (which remove the
temporary directory afterwards).
Add `create_json_files()` which creates a set of valid JSON files in a
temporary directory using the `models.OutputPackageBase` model.

repo_management/convert.py:
Add the class `RepoDbFile` which is used to initialize a jinja
environment and allows to render the 'desc.j2' and 'files.j2' templates.
Add `_desc_data_line_to_dicts()` which allows for assigning the data
from a line in a 'desc' file to be assigned to a dict, that corresponds
to the line's designated data type.
Simplify `_desc_data_to_model()` by making use of
`_desc_data_line_to_dicts()`.
Change `_desc_data_to_model()` by guarding against pydantic
ValidationErrors and raising a RepoManagementValidationError instead.

tests/test_convert.py:
Add tests for `RepoDbFile`.

repo_management/defaults.py:
Add the IntEnum `RepoDbType` to be able to identify/address the
different types of binary database files (e.g. .db vs. .files).
Change the %LICENSE% entry in `DESC_JSON` to not track the name in
plural (use "license" instead of "licenses") to stay in line with the
naming of the original variable.
Add `DB_USER`, `DB_GROUP`, `DB_FILE_MODE`, `DB_DIR_MODE`, to provide
defaults for which user and group to create files and directories with
which file mode for, when generating a database file.

repo_management/models.py:
Change the `License` model to track an optional list of license
identifiers using the singular term ("license" instead of "licenses") to
have more predictable and coherent naming of variables and attributes.
Add `Base` to `OutputPackageBase` as this way the model is more complete
and it becomes much easier to use it, without having to also pass the
pkgbase alongside it.
Add the convenience method `get_packages_as_models()` to
`OutputPackageBase` which returns the list of packages as tuples of
`PackageDesc` and `Files`.
Extend `PackageDesc` by the convenience method `get_output_package()` to
easily convert an instance of `PackageDesc` and an optional instance of
`Files` to an instance of `OutputPackage`.
Move the declaration of `OutputPackage` above that of `PackageDesc` so
that `PackageDesc.get_output_package()` can declare typing.

tests/test_models.py
Add tests for `PackageDesc.get_output_package()` and
`OutputPackageBase.get_packages_as_models()`.

repo_management/errors.py:
Add the Exception class `RepoManagementError` serving as a base, from
which `RepoManagementFileError` is derived, which serves as a general
Error class to signal problems with file interaction.
Add the Exception class `RepoManagementFileNotFoundError`, which is
derived from `RepoManagementFileError`.
Add `RepoManagementValidationError` which derives from
`RepoManagementError` and is raised on validation issues of file
data.

poetry.lock:
Add the current version of jinja2 to the lockfile.

pyproject.toml:
Add jinja2 as repository dependency.

repo_management/templates/{desc,files}.j2:
Add jinja2 templates for the 'desc' and 'files' files in a repository
database (standard sync db or files sync db).

db2json/*:
Remove obsolete db2json script and README.
It has been superseded by the script definition in pyproject.toml.

pyproject.toml:
Add a script setup for dbj2son pointing to `repo_management.cli:db2json`.
This can be used via `poetry run db2json`.

repo_management/cli.py:
Add `db2json()` which may serve as an entrypoint for a db2json script,
as it makes use of the argument parser defined in
`argparse.ArgParseFactory.db2json()` and calls
`operations.dump_db_to_json_files()` using the provided inputs.

tests/test_cli.py:
Add a unittest for `db2json()`, which patches all called methods.
The methods are tested extensively elsewhere. However, the function
should see proper integration testing via this endpoint in the future.

repo_management/operations.py:
Remove the explicit Path checks from `dump_db_to_json_files()` as they
are now generically implemented in argparse.ArgParseFactory.

tests/test_operations.py
Remove tests for Path checks in `dump_db_to_json_files()`.

repo_management/argparse.py:
Add `ArgParseFactory`, which serves as an argparse.ArgumentParser
factory (new variants are implemented as classmethods).
The class provides a generic constructor, that offers a default parser
with a verbose argument and a description, alongside a specific one for
the db2json script.
Additionally, helper methods are provided to transform strings to Path
instances, while ensuring that they are either an existing file or
directory.

tests/test_argparse.py:
Add tests for all of `ArgParseFactory`.

repo_management/operations.py:
Add `dump_db_to_json_files()` which allows for a repository database
file to be read and all of its members to be serialized as JSON and
written to files based upon the packages respective pkgbases.
For the JSON serialization orjson is used, as it provides the highest
speeds available and seems the most correct implementation.

tests/test_operations.py:
Add a fixture to create and destroy a temporary directory (used to write
JSON files to).
Add tests for `dump_db_to_json_files()`.

poetry.lock:
Update lock file for current versions of orjson and mock.

pyproject.toml:
Add orjson to the list of dependencies.
Add mock to the list of development dependencies.

tests/test_files.py:
Make use of common helper in `tests/fixtures.py` for creating repository
database fixtures.