repod issueshttps://gitlab.archlinux.org/archlinux/repod/-/issues2022-12-10T08:36:12Zhttps://gitlab.archlinux.org/archlinux/repod/-/issues/159Create a parser for .SRCINFO files2022-12-10T08:36:12ZDavid RungeCreate a parser for .SRCINFO filesTo enable us to validate data of an upstream package source repository, we require a parser for `.SRCINFO` files.To enable us to validate data of an upstream package source repository, we require a parser for `.SRCINFO` files.0.3.0 - workflows and checksDavid RungeDavid Rungehttps://gitlab.archlinux.org/archlinux/repod/-/issues/158Add configurable check to dissallow unsafe checksum algorithms in source info...2023-07-17T19:54:04ZDavid RungeAdd configurable check to dissallow unsafe checksum algorithms in source informationWe want to ensure that we can disallow unsafe checksum algorithms (e.g. SHA-1, MD5) in the package sources.
This can be ensured by checking out the package source repo and parsing its .SRCINFO file.We want to ensure that we can disallow unsafe checksum algorithms (e.g. SHA-1, MD5) in the package sources.
This can be ensured by checking out the package source repo and parsing its .SRCINFO file.workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/157Add configurable check to ensure that we have built with a specific build too...2023-07-17T19:54:04ZDavid RungeAdd configurable check to ensure that we have built with a specific build toolingWe need a configurable check to ensure that we have built a package using a specific build tooling (e.g. `devtools`).
This information is available in the `.BUILDINFO` file of each package.We need a configurable check to ensure that we have built a package using a specific build tooling (e.g. `devtools`).
This information is available in the `.BUILDINFO` file of each package.workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/156Implement local signing of repository sync databases2023-07-17T19:54:04ZDavid RungeImplement local signing of repository sync databasesUnlike #32, this ticket is about allowing users of repod's user-mode to sign repository sync databases.
This integration should grant the most secure avenue (i.e. smartcard integration).Unlike #32, this ticket is about allowing users of repod's user-mode to sign repository sync databases.
This integration should grant the most secure avenue (i.e. smartcard integration).workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/155Evaluate Git/ PGP integration for signing and signature checking2023-07-17T19:54:04ZDavid RungeEvaluate Git/ PGP integration for signing and signature checkingFor checking signatures against a list of allowed PGP key IDs (e.g. from pacman-key) and the local signing of repository sync databases, we need integration with tooling that can deal with PGP.
When looking at other avenues for signatur...For checking signatures against a list of allowed PGP key IDs (e.g. from pacman-key) and the local signing of repository sync databases, we need integration with tooling that can deal with PGP.
When looking at other avenues for signature validation in git repositories (e.g. #151 ), one could also look at openssh based signature integration (this would not be something that pacman can handle for repository sync databases though, so it falls flat in the use-case of repository sync database signing - #32).
## Requirements
| repod mode | git | smartcard (db signing) | git tag validation | WOT integration (git tag/ db validation) |
| --- | --- | --- | --- | --- |
| user | Yes | Yes | Yes | Yes [2] |
| system | Yes | Yes [1] | Yes | Yes [2] |
[1] If we want to allow a dedicated machine to be able to sign sync databases itself, it would probably be good to also have smartcard support here.
[2] Depending on support in keyringctl (see https://gitlab.archlinux.org/archlinux/keyringctl/-/issues/3), we can also evaulate the use of a flat file here (i.e. a keyring file with all "allowed keys") instead of hooking into an existing web-of-trust.
## PGP providers
### gpgme
* [pypi.org package](https://pypi.org/project/gpg/) is extremely out-of-date (last release 2018, 12(!) releases behind current gpgme)
* package does not build in venv (as the sdist is so outdated and uses some super ancient integration), only system integration can be used until upstream fixes this
* [patches have been provided to upstream](https://lists.gnupg.org/pipermail/gnupg-devel/2022-November/035158.html) for fixing sdist tarball generation and PEP517 based builds (so far no response)
### sequoia
* [no sdist tarballs or wheels on pypi.org](https://gitlab.com/sequoia-pgp/sequoia-ffi/-/issues/23), only system integration can be used until upstream fixes this
* unclear feature set
* upstream proposes using pyo3 to build something custom or [johnnycanencrypt](https://github.com/kushaldas/johnnycanencrypt) instead
### johnnycanencrypt
* provides sdist tarballs and wheels on [pypi.org](https://pypi.org/project/johnnycanencrypt/)
* written in rust, based on sequoia
* basic, does not integrate with keyring, requires a local "keystore" directory
* seems to have smartcard integration
* does not yet use sequoia's internal keystore implementation/specification
### python-gnupg
* has sdist tarballs and wheels on pypi.org
* deveveloped at https://github.com/vsajip/python-gnupg
* shells out to gpg :cry:
### pgpy
* has sdist tarballs and wheels on pypi.org
* developed at: https://github.com/SecurityInnovation/PGPy
* PGP implementation in Python
* feature set needs evaluation
## Git providers
### dulwich
* somehow [integrates with gpg](https://github.com/jelmer/dulwich/blob/82476465134520952ccb2f3b56a10bf431629725/setup.cfg#L37) for signatures (unclear if they ever test this, given that everything is terribly out-of-date and broken)
### pygit2
* pygit2 can get to signature strings, so PGP integration is likely open/ can be implemented whichever way
### gitpython
* long track record of botched releases (faith in upstream quite low also given that the project seems to be in maintenance-only mode over developing gitoxide)
* relies on system git and gnupg executablesworkflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/154Add documentation on archive directory structure2023-07-17T19:54:03ZDavid RungeAdd documentation on archive directory structureWhen creating the archiving MR !137 I did not create a documentation page about the archive directory structure.
This should still be added, so that the overall structure (as it exists right now) is explained.When creating the archiving MR !137 I did not create a documentation page about the archive directory structure.
This should still be added, so that the overall structure (as it exists right now) is explained.workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/153Improve readability of man page2022-12-05T21:28:35ZDavid RungeImprove readability of man pageCurrently the man page for `repod.conf` uses a lot of list elements, which is unreadible.
Additionally, the option intros should be switched to `option = ` instead of `option:` to indicate correct use of option to the user (while the opt...Currently the man page for `repod.conf` uses a lot of list elements, which is unreadible.
Additionally, the option intros should be switched to `option = ` instead of `option:` to indicate correct use of option to the user (while the option description should start one line below the option).0.3.0 - workflows and checksDavid RungeDavid Rungehttps://gitlab.archlinux.org/archlinux/repod/-/issues/152Improve non-debug error output of CLI tooling2022-11-17T12:14:33ZDavid RungeImprove non-debug error output of CLI toolingCurrently only the debug output of the CLI will emit the actual error message.
The error message from a Task/Check should be bubbled up instead, so that it can be emitted also in non-debug mode.Currently only the debug output of the CLI will emit the actual error message.
The error message from a Task/Check should be bubbled up instead, so that it can be emitted also in non-debug mode.https://gitlab.archlinux.org/archlinux/repod/-/issues/151Attest upstream metadata2023-07-17T19:54:03ZDavid RungeAttest upstream metadataWhen adding a package we want to be able to configure the attestation of upstream source metadata.
Metadata has diverse sources in the upstream source repository:
- PKGBUILD (e.g. pkgbase, pkgname, pkgver, epoch, pkgrel)
- PKGBUILD chec...When adding a package we want to be able to configure the attestation of upstream source metadata.
Metadata has diverse sources in the upstream source repository:
- PKGBUILD (e.g. pkgbase, pkgname, pkgver, epoch, pkgrel)
- PKGBUILD checksum (see #120)
- git tag exists (see #118)
- git tag is signed by valid packager (e.g. UID and signature match those in pacman-key or a custom list of PGP key IDs)
The above requirements have direct influence on #120 and #118 and it appears it will make most sense to not implement source upstream handling per source forge but rather via a dedicated git backend, as we will require (long-living) checkouts for the attestations.workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/150Add configurable check to ensure that installed dependencies of packages in a...2022-11-23T13:42:33ZLevente Polyakanthraxx@archlinux.orgAdd configurable check to ensure that installed dependencies of packages in a transaction are availableThis is required for reproducible builds to ensure that any listed dependency in its used version is available in the repo, pool or current transaction otherwise it will not be possible to satisfy the install requirements during a reprod...This is required for reproducible builds to ensure that any listed dependency in its used version is available in the repo, pool or current transaction otherwise it will not be possible to satisfy the install requirements during a reproducible builds.
To achieve this we need to go through the `.BUILDINFO` file and extract all `installed` fields (pkgname + exact version).
Once collected we need to check them against availability. If one of the dependencies in the `installed` fields is not available in that specified version in the package pool or the archive, deny the release of that package.
Slightly related to #142, but #142 is meant to ensure consistency within a given repo (f.e. `[core]` only depends on `[core]` packages)0.3.0 - workflows and checksDavid RungeDavid Rungehttps://gitlab.archlinux.org/archlinux/repod/-/issues/149Define repository groups2022-12-03T18:47:49ZDavid RungeDefine repository groupsIn relation to #145 we require the notion of "repositories belonging together".
This allows us to check whether e.g. packages must be unique in a set of repositories (e.g. core, extra, community must not contain the same pkgbase/pkgnames).In relation to #145 we require the notion of "repositories belonging together".
This allows us to check whether e.g. packages must be unique in a set of repositories (e.g. core, extra, community must not contain the same pkgbase/pkgnames).0.3.0 - workflows and checksDavid RungeDavid Rungehttps://gitlab.archlinux.org/archlinux/repod/-/issues/148Add differences in functionality between repod and other systems to documenta...2023-07-17T19:54:03ZDavid RungeAdd differences in functionality between repod and other systems to documentationWe should define a feature matrix in documentation that outlines the differences between repod and other systems for managing package repositories (e.g. [dbscripts](https://gitlab.archlinux.org/archlinux/dbscripts) or [ahriman](https://g...We should define a feature matrix in documentation that outlines the differences between repod and other systems for managing package repositories (e.g. [dbscripts](https://gitlab.archlinux.org/archlinux/dbscripts) or [ahriman](https://github.com/arcan1s/ahriman)).workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/147Importing a package can be very slow with a big management repo2022-12-16T10:47:20ZKristian KlausenImporting a package can be very slow with a big management repoI noticed that importing a ton of packages in [my management repo](https://gitlab.archlinux.org/klausenbusk/management-repo) can be [very slow](https://gitlab.archlinux.org/klausenbusk/management-repo/-/jobs/97748) (a new management repo...I noticed that importing a ton of packages in [my management repo](https://gitlab.archlinux.org/klausenbusk/management-repo) can be [very slow](https://gitlab.archlinux.org/klausenbusk/management-repo/-/jobs/97748) (a new management repo is created for new packages and it is merged into the existing management repo when the importing is done).
Some quick benchmarking:
0 existing packages:
```sh
$ time pdm run repod-file -c ../repod.conf repo importpkg bash-5.1.016-1-x86_64.pkg.tar.zst community
real 0m1,164s
user 0m1,013s
sys 0m0,143s
```
100 existing packages:
```sh
time pdm run repod-file -c ../repod.conf repo importpkg bash-5.1.016-1-x86_64.pkg.tar.zst community
real 0m4,552s
user 0m4,377s
sys 0m0,154s
```
1000 existing packages:
```sh
time pdm run repod-file -c ../repod.conf repo importpkg bash-5.1.016-1-x86_64.pkg.tar.zst community
real 0m25,897s
user 0m25,299s
sys 0m0,458s
```
5000 existing packages:
```sh
time pdm run repod-file -c ../repod.conf repo importpkg bash-5.1.016-1-x86_64.pkg.tar.zst community
real 2m29,775s
user 2m27,144s
sys 0m1,772s
```https://gitlab.archlinux.org/archlinux/repod/-/issues/146Fix flaky file detection2022-10-27T07:42:18ZDavid RungeFix flaky file detectionIt appears file is sometimes a bit flaky:
* https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=876316
* https://dl.klausen.dk/shots/ko1FokNUVRtRdOk1bQjv7ygff0yV2zlx.txt
It may wrongly detect some files as `DOS/MBR boot sector`.It appears file is sometimes a bit flaky:
* https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=876316
* https://dl.klausen.dk/shots/ko1FokNUVRtRdOk1bQjv7ygff0yV2zlx.txt
It may wrongly detect some files as `DOS/MBR boot sector`.https://gitlab.archlinux.org/archlinux/repod/-/issues/145Implement cross-repository bulk actions2023-07-17T19:54:03ZDavid RungeImplement cross-repository bulk actionsWhen looking at the use-case of e.g. moving packages from two testing repositories to their respective stable counterparts, we want to have a way of doing this in one atomic action.
The [workflows](https://gitlab.archlinux.org/archlinux...When looking at the use-case of e.g. moving packages from two testing repositories to their respective stable counterparts, we want to have a way of doing this in one atomic action.
The [workflows](https://gitlab.archlinux.org/archlinux/repod/-/blob/9d2fcc6e4df59107f8dfb172fbd5c93d5f6c7e46/repod/action/workflow.py) currently implemented can only be used for bulk actions on a single repository.workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/144Add workflow scenarios to documentation2023-07-17T19:54:03ZDavid RungeAdd workflow scenarios to documentationFrom a contributor and user perspective it would be helpful to have a well-defined workflow (i.e. a cleaned up version of #49).From a contributor and user perspective it would be helpful to have a well-defined workflow (i.e. a cleaned up version of #49).workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/143Add configurable check to ensure that makedepends, checkdepends and optdepend...2023-07-17T19:54:03ZDavid RungeAdd configurable check to ensure that makedepends, checkdepends and optdepends can be metA cluster of repositories in the same management repo should be considered as packages for the same purpose (e.g. a distribution).
A configurable check needs to be added, that ensures the makedepends, checkdepends and optdepends of a new...A cluster of repositories in the same management repo should be considered as packages for the same purpose (e.g. a distribution).
A configurable check needs to be added, that ensures the makedepends, checkdepends and optdepends of a new package can be met within the context of the repositories (also when adding several packages at once).workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/142Add configurable check to ensure that repositories are self-contained2023-07-17T19:54:03ZDavid RungeAdd configurable check to ensure that repositories are self-containedSome repositories have stricter requirements and require, that they are self-contained (any added package can only depend on packages also in that repository).
This should become a configurable option for each repository.Some repositories have stricter requirements and require, that they are self-contained (any added package can only depend on packages also in that repository).
This should become a configurable option for each repository.workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/141Add workflow to move pkgbases2023-07-17T19:54:03ZDavid RungeAdd workflow to move pkgbasesWe need to create a workflow function to move pkgbases from one repository (or one stability layer) to another.
This must ensure, that symlinks in management repositories and package repositories are created accordingly and that we do n...We need to create a workflow function to move pkgbases from one repository (or one stability layer) to another.
This must ensure, that symlinks in management repositories and package repositories are created accordingly and that we do not permit moving against the stability flow (e.g. no move from stable to testing).workflowshttps://gitlab.archlinux.org/archlinux/repod/-/issues/140Add workflow to remove pkgbases2023-07-17T19:54:03ZDavid RungeAdd workflow to remove pkgbasesWe need to create a workflow function to remove a list of pkgbases from a repository.
This workflow must entail to remove symlinks in the management repository and package repository accordingly.We need to create a workflow function to remove a list of pkgbases from a repository.
This workflow must entail to remove symlinks in the management repository and package repository accordingly.workflowsDavid RungeDavid Runge