- 24 Mar, 2021 1 commit
-
-
David Runge authored
repo_management/models.py: Add validators to the `BuildDate`, `CSize`, `ISize`, `Name` and `Version` models to ensure positive integers, correct names and versions. Add the comparator instance methods `Version.is_older_than()` and `Version.is_newer_than()` which rely on pyalpm's `vercmp()` to compare two version strings and return whether they are older or newer (respectively). Update the documentation for `ISize` to reflect, that it represents size of an installed package. tests/test_models.py: Add tests for added validators and comparator methods.
-
- 22 Mar, 2021 1 commit
-
-
David Runge authored
repo_management/models.py: Document all models and sort the single attribute models alphabetically for easier readibility. Change the `License` model to describe a required attribute instead of an optional (all packages need a license). Remove the unused model `PackageFiles`.
-
- 20 Mar, 2021 3 commits
-
-
David Runge authored
repo_management/cli.py: Change `db2json` and `json2db` to try/except `errors.RepoManagementError` and `argpase.ArgumentTypeError` and exit with a non-zero return code in that case. tests/test_cli.py: Change the tests for `db2json` and `json2db` to cover potentially raising `errors.RepoManagementError` and `argpase.ArgumentTypeError` and checking whether exiting with a non-zero return code in that case. Add parametrized integration test `test_transform_databases()` that transforms existing pacman .files sync databases to JSON files and back to .files repository databases again. Add parametrized integration `test_transform_databases_and_use_with_pacman()` to transform existing pacman .files sync databases to JSON files and create .files and .db repository databases from them. Afterwards pacman uses the created repository databases to search for packages with the "linux" keyword in them and list the files owned by the "linux" package using the created .files sync databases.
-
David Runge authored
repo_management/convert.py: Change the docuemntation of `_desc_data_line_to_dicts()` to reflect, that it may raise ValueError under certain conditions. Change `_desc_data_to_model()` to try/except ValueError raised by `_desc_data_line_to_dicts()` and raise a `errors.REpomanagementValidationError` instead. tests/test_convert.py: Change tests expecting to raise `ValueError` to expect `errors.RepoManagementValidationError` instead.
-
David Runge authored
repo_management/commands.py: Add `run_command()` which allows for running an external command with the help of subprocess-tee. Add the private method `_print_env()` which is used to print the optional environment variables given to `run_command()`. tests/test_commands.py: Add tests for `_print_env()` and `run_command()`.
-
- 18 Mar, 2021 6 commits
-
-
David Runge authored
repo_management/cli.py: Add `json2db()` as entrypoint for the `json2db` script. tests/test_cli.py: Add a test for `json2db()`.
-
David Runge authored
repo_management/argparse.py: Add `ArgParseFactory.json2db()` as an ArgumentParser for the `json2db` script. Add `ArgParseFactory.string_to_writable_file_path()` for returning a Path object, that is guaranteed to be a writable file. tests/test_argparse.py: Add tests for `ArgParseFactory.{json2db,string_to_writable_file_path}()`.
-
David Runge authored
repo_management/operations.py: Add `create_db_from_json_files()` to create a repository database from JSON files in a directory. Change `db_file_as_models()` to rely on `models.PackageDesc.get_output_packages()` instead of `convert._transform_package_desc_to_output_package()` and add a missing parameter (base) to the instantiation of `models.OutputPackageBase()`. tests/test_operations.py: Add the two fixtures `dummy_json_files_in_dir()` and `empty_file()` to create a few (valid) dummy input JSON files in a temporary directory and an empty file (respectively). Add a test for `create_db_from_json_files()`.
-
David Runge authored
repo_management/files.py: Add `_json_files_in_directory()` yield a list of files with a .json suffix in a directory. Add `_read_pkgbase_json_file()` to read a JSON file representing a pkgbase and return an `OutputPackageBase` model. Add `_write_db_file()` to open a tar file for writing. Add `_stream_package_base_to_db()` to stream a `models.OutputPackage()` to a tar file (which represents a .db or a .files repository database). tests/test_files.py: Add tests for `_json_files_in_directory()`, `_read_pkgbase_json_file()`, `_write_db_file()` and `_stream_package_base_to_db()`.
-
David Runge authored
repo_management/convert.py: Add the class `RepoDbFile` which is used to initialize a jinja environment and allows to render the 'desc.j2' and 'files.j2' templates. Add `_desc_data_line_to_dicts()` which allows for assigning the data from a line in a 'desc' file to be assigned to a dict, that corresponds to the line's designated data type. Simplify `_desc_data_to_model()` by making use of `_desc_data_line_to_dicts()`. Change `_desc_data_to_model()` by guarding against pydantic ValidationErrors and raising a RepoManagementValidationError instead. tests/test_convert.py: Add tests for `RepoDbFile`.
-
David Runge authored
repo_management/defaults.py: Add the IntEnum `RepoDbType` to be able to identify/address the different types of binary database files (e.g. .db vs. .files). Change the %LICENSE% entry in `DESC_JSON` to not track the name in plural (use "license" instead of "licenses") to stay in line with the naming of the original variable. Add `DB_USER`, `DB_GROUP`, `DB_FILE_MODE`, `DB_DIR_MODE`, to provide defaults for which user and group to create files and directories with which file mode for, when generating a database file. repo_management/models.py: Change the `License` model to track an optional list of license identifiers using the singular term ("license" instead of "licenses") to have more predictable and coherent naming of variables and attributes. Add `Base` to `OutputPackageBase` as this way the model is more complete and it becomes much easier to use it, without having to also pass the pkgbase alongside it. Add the convenience method `get_packages_as_models()` to `OutputPackageBase` which returns the list of packages as tuples of `PackageDesc` and `Files`. Extend `PackageDesc` by the convenience method `get_output_package()` to easily convert an instance of `PackageDesc` and an optional instance of `Files` to an instance of `OutputPackage`. Move the declaration of `OutputPackage` above that of `PackageDesc` so that `PackageDesc.get_output_package()` can declare typing. tests/test_models.py Add tests for `PackageDesc.get_output_package()` and `OutputPackageBase.get_packages_as_models()`.
-
- 17 Mar, 2021 1 commit
-
-
David Runge authored
repo_management/errors.py: Add the Exception class `RepoManagementError` serving as a base, from which `RepoManagementFileError` is derived, which serves as a general Error class to signal problems with file interaction. Add the Exception class `RepoManagementFileNotFoundError`, which is derived from `RepoManagementFileError`. Add `RepoManagementValidationError` which derives from `RepoManagementError` and is raised on validation issues of file data.
-
- 15 Mar, 2021 1 commit
-
-
David Runge authored
repo_management/templates/{desc,files}.j2: Add jinja2 templates for the 'desc' and 'files' files in a repository database (standard sync db or files sync db).
-
- 09 Mar, 2021 3 commits
-
-
David Runge authored
repo_management/cli.py: Add `db2json()` which may serve as an entrypoint for a db2json script, as it makes use of the argument parser defined in `argparse.ArgParseFactory.db2json()` and calls `operations.dump_db_to_json_files()` using the provided inputs. tests/test_cli.py: Add a unittest for `db2json()`, which patches all called methods. The methods are tested extensively elsewhere. However, the function should see proper integration testing via this endpoint in the future.
-
David Runge authored
repo_management/operations.py: Remove the explicit Path checks from `dump_db_to_json_files()` as they are now generically implemented in argparse.ArgParseFactory. tests/test_operations.py Remove tests for Path checks in `dump_db_to_json_files()`.
-
David Runge authored
repo_management/argparse.py: Add `ArgParseFactory`, which serves as an argparse.ArgumentParser factory (new variants are implemented as classmethods). The class provides a generic constructor, that offers a default parser with a verbose argument and a description, alongside a specific one for the db2json script. Additionally, helper methods are provided to transform strings to Path instances, while ensuring that they are either an existing file or directory. tests/test_argparse.py: Add tests for all of `ArgParseFactory`.
-
- 08 Mar, 2021 7 commits
-
-
David Runge authored
repo_management/operations.py: Add `dump_db_to_json_files()` which allows for a repository database file to be read and all of its members to be serialized as JSON and written to files based upon the packages respective pkgbases. For the JSON serialization orjson is used, as it provides the highest speeds available and seems the most correct implementation. tests/test_operations.py: Add a fixture to create and destroy a temporary directory (used to write JSON files to). Add tests for `dump_db_to_json_files()`.
-
David Runge authored
repo_management/operations.py: Add the public method `db_file_as_models()`, which yields all members of a repository database as a Tuple of the pkgbase name and the pkgbase data (represented as an instance of models.OutputPackageBase). tests/test_operations.py: Add a test for `db_file_as_models()`.
-
David Runge authored
repo_management/files.py: Add the private method `_extract_db_member_package_name()` which can be used to extract the package name from the name of a member of repository database. Rename `_read_db_file_member()` to `_db_file_member_as_model()` and change the method to yield a `models.RepoDbMemberData` instead of a `tarfile.TarInfo` to provide actual data (and required context) to callers of the method. tests/test_file.py: Add parametrized test for `_extract_db_member_package_name()`. Change test for `_db_file_member_as_model()` to assert the correct return type.
-
David Runge authored
repo_management/convert.py: Add private method `_transform_package_desc_to_output_package()` which can be used to construct a `models.OutputPackage` from a `models.PackageDesc` (and optionally a `models.PackageFiles`). The method drops attributes from a dict representation of the `models.PackageDesc` instance, that are not relevant for the `models.OutputPackage`. tests/test_convert.py: Add parametrized test for `_transform_package_desc_to_output_package()`.
-
David Runge authored
repo_management/models.py: Add model `PackageFiles` to track the list of files for a given package. Add model `RepoDbMemberType` to track the type of a repository database member (distinguished by the IntEnum defaults.RepoDbMemberType). Add model `RepoDbMemberData` to track the data (represented by an io.StringIO instance) of a repository database member. Add model `OutputPackage` to track the attributes required for a single package in the context of a representation within its pkgbase. Add model `OutputPackageBase` to track the attributes required for a pkgbase and its list of packages.
-
David Runge authored
repo_management/defaults.py: Add the IntEnum `RepoDbMemberType` to distinguish different members (files) in a repository database (e.g. 'desc' or 'files' files).
-
David Runge authored
repo_management/files.py: Add iterator method `_read_db_file_member()` to yield the members of a provided tarfile.TarFile. The members are filtered by name using a regular expression, which can be overridden. tests/test_file.py: Change `create_db_file()` to also create 'desc' and 'files' files below the created temporary directories. Assert, that all tested methods return a value. Add test for `_read_db_file_member()`.
-
- 07 Mar, 2021 3 commits
-
-
David Runge authored
repo_management/files.py: Add `_read_db_file()` to open a db file and return it as tarfile.Tarfile. The method assumes gzip-compressed databases by default, but can be used with other compression algorithms. tests/test_file.py: Add fixtures and tests for `_read_db_file()`.
-
David Runge authored
repo_management/defaults.py: Add the IntEnmu `FieldType` which tracks the required type for an attribute when converting from 'desc' or 'files' files to pydantic models. Extend the DESC_JSON and FILES_JSON globals by introducing a dict per key, that describes the name and the required type, when converting to pydantic models. repo_management/convert.py: Rename `_files_data_to_dict()` to `_read_files_data_to_model()` and return a pydantic model (`models.Files`) instead of a custom dict. Add `_desc_data_to_model()` which allows for conversion of a 'desc' data file (represented as io.StringIO) to a pydantic model (`models.PackageDesc`). tests/test_convert.py: Add another test for raising RuntimeError in `_read_files_data_to_model()`. Add a parametrized test for `_desc_data_to_model()`.
-
David Runge authored
repo_management/models.py: Add pydantic models for all headers in 'desc' and 'files' files (found in a repository database per pkgbase). The models describe the attribute names in use when reading data from the respective files and whether they are required or not.
-
- 06 Mar, 2021 1 commit
-
-
David Runge authored
repo_management/defaults.py: Add the basic defaults module, providing data that translates keywords found in the 'desc' and 'files' files of a binary repository database to those used in JSON files, representing the same data. repo_management/convert.py: Add the convert module, and introduce the simple private method (reimplementing a subset of the features from db2json) `_files_data_to_dict()`, which translates the contents of a `files` file to a typed dict.
-