repod issueshttps://gitlab.archlinux.org/archlinux/repod/-/issues2023-07-17T19:54:04Zhttps://gitlab.archlinux.org/archlinux/repod/-/issues/158Add configurable check to dissallow unsafe checksum algorithms in source info...2023-07-17T19:54:04ZDavid RungeAdd configurable check to dissallow unsafe checksum algorithms in source informationWe want to ensure that we can disallow unsafe checksum algorithms (e.g. SHA-1, MD5) in the package sources.
This can be ensured by checking out the package source repo and parsing its .SRCINFO file.We want to ensure that we can disallow unsafe checksum algorithms (e.g. SHA-1, MD5) in the package sources.
This can be ensured by checking out the package source repo and parsing its .SRCINFO file.workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/157Add configurable check to ensure that we have built with a specific build too...2023-07-17T19:54:04ZDavid RungeAdd configurable check to ensure that we have built with a specific build toolingWe need a configurable check to ensure that we have built a package using a specific build tooling (e.g. `devtools`).
This information is available in the `.BUILDINFO` file of each package.We need a configurable check to ensure that we have built a package using a specific build tooling (e.g. `devtools`).
This information is available in the `.BUILDINFO` file of each package.workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/156Implement local signing of repository sync databases2023-07-17T19:54:04ZDavid RungeImplement local signing of repository sync databasesUnlike #32, this ticket is about allowing users of repod's user-mode to sign repository sync databases.
This integration should grant the most secure avenue (i.e. smartcard integration).Unlike #32, this ticket is about allowing users of repod's user-mode to sign repository sync databases.
This integration should grant the most secure avenue (i.e. smartcard integration).workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/155Evaluate Git/ PGP integration for signing and signature checking2023-07-17T19:54:04ZDavid RungeEvaluate Git/ PGP integration for signing and signature checkingFor checking signatures against a list of allowed PGP key IDs (e.g. from pacman-key) and the local signing of repository sync databases, we need integration with tooling that can deal with PGP.
When looking at other avenues for signatur...For checking signatures against a list of allowed PGP key IDs (e.g. from pacman-key) and the local signing of repository sync databases, we need integration with tooling that can deal with PGP.
When looking at other avenues for signature validation in git repositories (e.g. #151 ), one could also look at openssh based signature integration (this would not be something that pacman can handle for repository sync databases though, so it falls flat in the use-case of repository sync database signing - #32).
## Requirements
| repod mode | git | smartcard (db signing) | git tag validation | WOT integration (git tag/ db validation) |
| --- | --- | --- | --- | --- |
| user | Yes | Yes | Yes | Yes [2] |
| system | Yes | Yes [1] | Yes | Yes [2] |
[1] If we want to allow a dedicated machine to be able to sign sync databases itself, it would probably be good to also have smartcard support here.
[2] Depending on support in keyringctl (see https://gitlab.archlinux.org/archlinux/keyringctl/-/issues/3), we can also evaulate the use of a flat file here (i.e. a keyring file with all "allowed keys") instead of hooking into an existing web-of-trust.
## PGP providers
### gpgme
* [pypi.org package](https://pypi.org/project/gpg/) is extremely out-of-date (last release 2018, 12(!) releases behind current gpgme)
* package does not build in venv (as the sdist is so outdated and uses some super ancient integration), only system integration can be used until upstream fixes this
* [patches have been provided to upstream](https://lists.gnupg.org/pipermail/gnupg-devel/2022-November/035158.html) for fixing sdist tarball generation and PEP517 based builds (so far no response)
### sequoia
* [no sdist tarballs or wheels on pypi.org](https://gitlab.com/sequoia-pgp/sequoia-ffi/-/issues/23), only system integration can be used until upstream fixes this
* unclear feature set
* upstream proposes using pyo3 to build something custom or [johnnycanencrypt](https://github.com/kushaldas/johnnycanencrypt) instead
### johnnycanencrypt
* provides sdist tarballs and wheels on [pypi.org](https://pypi.org/project/johnnycanencrypt/)
* written in rust, based on sequoia
* basic, does not integrate with keyring, requires a local "keystore" directory
* seems to have smartcard integration
* does not yet use sequoia's internal keystore implementation/specification
### python-gnupg
* has sdist tarballs and wheels on pypi.org
* deveveloped at https://github.com/vsajip/python-gnupg
* shells out to gpg :cry:
### pgpy
* has sdist tarballs and wheels on pypi.org
* developed at: https://github.com/SecurityInnovation/PGPy
* PGP implementation in Python
* feature set needs evaluation
## Git providers
### dulwich
* somehow [integrates with gpg](https://github.com/jelmer/dulwich/blob/82476465134520952ccb2f3b56a10bf431629725/setup.cfg#L37) for signatures (unclear if they ever test this, given that everything is terribly out-of-date and broken)
### pygit2
* pygit2 can get to signature strings, so PGP integration is likely open/ can be implemented whichever way
### gitpython
* long track record of botched releases (faith in upstream quite low also given that the project seems to be in maintenance-only mode over developing gitoxide)
* relies on system git and gnupg executablesworkflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/154Add documentation on archive directory structure2023-07-17T19:54:03ZDavid RungeAdd documentation on archive directory structureWhen creating the archiving MR !137 I did not create a documentation page about the archive directory structure.
This should still be added, so that the overall structure (as it exists right now) is explained.When creating the archiving MR !137 I did not create a documentation page about the archive directory structure.
This should still be added, so that the overall structure (as it exists right now) is explained.workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/152Improve non-debug error output of CLI tooling2022-11-17T12:14:33ZDavid RungeImprove non-debug error output of CLI toolingCurrently only the debug output of the CLI will emit the actual error message.
The error message from a Task/Check should be bubbled up instead, so that it can be emitted also in non-debug mode.Currently only the debug output of the CLI will emit the actual error message.
The error message from a Task/Check should be bubbled up instead, so that it can be emitted also in non-debug mode.https://gitlab.archlinux.org/archlinux/repod/-/issues/151Attest upstream metadata2023-07-17T19:54:03ZDavid RungeAttest upstream metadataWhen adding a package we want to be able to configure the attestation of upstream source metadata.
Metadata has diverse sources in the upstream source repository:
- PKGBUILD (e.g. pkgbase, pkgname, pkgver, epoch, pkgrel)
- PKGBUILD chec...When adding a package we want to be able to configure the attestation of upstream source metadata.
Metadata has diverse sources in the upstream source repository:
- PKGBUILD (e.g. pkgbase, pkgname, pkgver, epoch, pkgrel)
- PKGBUILD checksum (see #120)
- git tag exists (see #118)
- git tag is signed by valid packager (e.g. UID and signature match those in pacman-key or a custom list of PGP key IDs)
The above requirements have direct influence on #120 and #118 and it appears it will make most sense to not implement source upstream handling per source forge but rather via a dedicated git backend, as we will require (long-living) checkouts for the attestations.workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/148Add differences in functionality between repod and other systems to documenta...2023-07-17T19:54:03ZDavid RungeAdd differences in functionality between repod and other systems to documentationWe should define a feature matrix in documentation that outlines the differences between repod and other systems for managing package repositories (e.g. [dbscripts](https://gitlab.archlinux.org/archlinux/dbscripts) or [ahriman](https://g...We should define a feature matrix in documentation that outlines the differences between repod and other systems for managing package repositories (e.g. [dbscripts](https://gitlab.archlinux.org/archlinux/dbscripts) or [ahriman](https://github.com/arcan1s/ahriman)).workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/147Importing a package can be very slow with a big management repo2022-12-16T10:47:20ZKristian KlausenImporting a package can be very slow with a big management repoI noticed that importing a ton of packages in [my management repo](https://gitlab.archlinux.org/klausenbusk/management-repo) can be [very slow](https://gitlab.archlinux.org/klausenbusk/management-repo/-/jobs/97748) (a new management repo...I noticed that importing a ton of packages in [my management repo](https://gitlab.archlinux.org/klausenbusk/management-repo) can be [very slow](https://gitlab.archlinux.org/klausenbusk/management-repo/-/jobs/97748) (a new management repo is created for new packages and it is merged into the existing management repo when the importing is done).
Some quick benchmarking:
0 existing packages:
```sh
$ time pdm run repod-file -c ../repod.conf repo importpkg bash-5.1.016-1-x86_64.pkg.tar.zst community
real 0m1,164s
user 0m1,013s
sys 0m0,143s
```
100 existing packages:
```sh
time pdm run repod-file -c ../repod.conf repo importpkg bash-5.1.016-1-x86_64.pkg.tar.zst community
real 0m4,552s
user 0m4,377s
sys 0m0,154s
```
1000 existing packages:
```sh
time pdm run repod-file -c ../repod.conf repo importpkg bash-5.1.016-1-x86_64.pkg.tar.zst community
real 0m25,897s
user 0m25,299s
sys 0m0,458s
```
5000 existing packages:
```sh
time pdm run repod-file -c ../repod.conf repo importpkg bash-5.1.016-1-x86_64.pkg.tar.zst community
real 2m29,775s
user 2m27,144s
sys 0m1,772s
```https://gitlab.archlinux.org/archlinux/repod/-/issues/145Implement cross-repository bulk actions2023-07-17T19:54:03ZDavid RungeImplement cross-repository bulk actionsWhen looking at the use-case of e.g. moving packages from two testing repositories to their respective stable counterparts, we want to have a way of doing this in one atomic action.
The [workflows](https://gitlab.archlinux.org/archlinux...When looking at the use-case of e.g. moving packages from two testing repositories to their respective stable counterparts, we want to have a way of doing this in one atomic action.
The [workflows](https://gitlab.archlinux.org/archlinux/repod/-/blob/9d2fcc6e4df59107f8dfb172fbd5c93d5f6c7e46/repod/action/workflow.py) currently implemented can only be used for bulk actions on a single repository.workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/144Add workflow scenarios to documentation2023-07-17T19:54:03ZDavid RungeAdd workflow scenarios to documentationFrom a contributor and user perspective it would be helpful to have a well-defined workflow (i.e. a cleaned up version of #49).From a contributor and user perspective it would be helpful to have a well-defined workflow (i.e. a cleaned up version of #49).workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/143Add configurable check to ensure that makedepends, checkdepends and optdepend...2023-07-17T19:54:03ZDavid RungeAdd configurable check to ensure that makedepends, checkdepends and optdepends can be metA cluster of repositories in the same management repo should be considered as packages for the same purpose (e.g. a distribution).
A configurable check needs to be added, that ensures the makedepends, checkdepends and optdepends of a new...A cluster of repositories in the same management repo should be considered as packages for the same purpose (e.g. a distribution).
A configurable check needs to be added, that ensures the makedepends, checkdepends and optdepends of a new package can be met within the context of the repositories (also when adding several packages at once).workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/142Add configurable check to ensure that repositories are self-contained2023-07-17T19:54:03ZDavid RungeAdd configurable check to ensure that repositories are self-containedSome repositories have stricter requirements and require, that they are self-contained (any added package can only depend on packages also in that repository).
This should become a configurable option for each repository.Some repositories have stricter requirements and require, that they are self-contained (any added package can only depend on packages also in that repository).
This should become a configurable option for each repository.workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/141Add workflow to move pkgbases2023-07-17T19:54:03ZDavid RungeAdd workflow to move pkgbasesWe need to create a workflow function to move pkgbases from one repository (or one stability layer) to another.
This must ensure, that symlinks in management repositories and package repositories are created accordingly and that we do n...We need to create a workflow function to move pkgbases from one repository (or one stability layer) to another.
This must ensure, that symlinks in management repositories and package repositories are created accordingly and that we do not permit moving against the stability flow (e.g. no move from stable to testing).workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/140Add workflow to remove pkgbases2023-07-17T19:54:03ZDavid RungeAdd workflow to remove pkgbasesWe need to create a workflow function to remove a list of pkgbases from a repository.
This workflow must entail to remove symlinks in the management repository and package repository accordingly.We need to create a workflow function to remove a list of pkgbases from a repository.
This workflow must entail to remove symlinks in the management repository and package repository accordingly.workflowsDavid RungeDavid Rungehttps://gitlab.archlinux.org/archlinux/repod/-/issues/133Add visualization of repod components for a top-level overview2022-10-12T18:09:27ZDavid RungeAdd visualization of repod components for a top-level overviewTo allow newcomers (and users) of the project to understand the workings of repod, we need to add a visual representation of data flow through the various components.
E.g. package -> binary repository & management repository -> sync data...To allow newcomers (and users) of the project to understand the workings of repod, we need to add a visual representation of data flow through the various components.
E.g. package -> binary repository & management repository -> sync database
This visualization can be done e.g. as an SVG file and be included in the documentation.https://gitlab.archlinux.org/archlinux/repod/-/issues/132Implement signoff system for packages2022-10-12T15:44:19ZDavid RungeImplement signoff system for packages Similar to how currently sign off is handled, we want to have a signoff system, which allows for authenticated users to sign off on packages in a repository's testing repo.
Depending on configuration, the repository should only allow m... Similar to how currently sign off is handled, we want to have a signoff system, which allows for authenticated users to sign off on packages in a repository's testing repo.
Depending on configuration, the repository should only allow moving of packages from testing to stable location if enough sign offs have been done.0.8.0 - create APIhttps://gitlab.archlinux.org/archlinux/repod/-/issues/129Support systemd-creds for any credentials2022-10-05T09:44:54ZDavid RungeSupport systemd-creds for any credentialsWhen supporting any form of credentials in configurations in the future, we also need to support [systemd-creds](https://man.archlinux.org/man/systemd-creds.1) to have a secure abstraction and allow not exposing secrets in configuration ...When supporting any form of credentials in configurations in the future, we also need to support [systemd-creds](https://man.archlinux.org/man/systemd-creds.1) to have a secure abstraction and allow not exposing secrets in configuration files when on a systemd based system.0.8.0 - create APIhttps://gitlab.archlinux.org/archlinux/repod/-/issues/125Implement handling of source tarballs for pkgbases2022-09-28T18:17:13ZDavid RungeImplement handling of source tarballs for pkgbasesWhen we are providing GPL packages, we need to also provide their sources, as the license requires us to do so.
For Arch Linux this is currently done by a [script](https://gitlab.archlinux.org/archlinux/dbscripts/-/blob/master/cron-jobs/...When we are providing GPL packages, we need to also provide their sources, as the license requires us to do so.
For Arch Linux this is currently done by a [script](https://gitlab.archlinux.org/archlinux/dbscripts/-/blob/master/cron-jobs/sourceballs), that runs on a timer and builds source tarballs for any package in the pool which is (partly) licensed under any of the GPL licenses.
Therefore we need to implement the handling of source tarballs in their respective repository. The settings module is already aware of source repository directories for each package repository.
Note: For license identification an overhaul of how licenses are currently being used is currently evaluated.https://gitlab.archlinux.org/archlinux/repod/-/issues/120Check PKGBUILD exists and matches sha256sum2023-07-17T19:54:02ZDavid RungeCheck PKGBUILD exists and matches sha256sumWhen adding or updating packages we need to ensure that the package's PKGBUILD sha256sum (in its .BUILDINFO file) matches the one from the upstream repository at the given tag.When adding or updating packages we need to ensure that the package's PKGBUILD sha256sum (in its .BUILDINFO file) matches the one from the upstream repository at the given tag.workflows