The script was disabled over 7 years ago and just a month after
everything related to ABS was removed[2] (which the script depends on).

[1] archlinux/infrastructure@bcd20f65
[2] archlinux/infrastructure@184905ed

With the archive and repos no longer hosted on the same server[1], we
can let archivetools[2] update the packages directory on the archive
server once a day and risk losing packages (if a package is updated
multiple times in the same day).

This is not ideal, so instead we will continue as before and let
db-archive store the archived packages locally and then we will transfer
them from the repos server to the archive server.

The transfer part is more easily done, if the archived package files are
copied in place in an atomic way on the repos server as we don't risk
transferring partially written files that way.

[1] https://lists.archlinux.org/archives/list/arch-dev-public@lists.archlinux.org/thread/B66SQUW4HOJAN2WHG4RLEBLYWNWOSF3Y/
[2] https://gitlab.archlinux.org/archlinux/archivetools/-/blob/d5cbd5a5a9454eecd5c46b64ebc08d5087a25f2d/archive.sh#L141-L200

With the archive and repos no longer hosted on the same server[1], the
existence of "dependent packages" can no longer be determined solely by
checking whether the file exists in the local archive directory, which
now contains at most a few days worth of packages.

To solve this the archive index[2] must also be checked which contains
all archived packages. We still need to check locally first, as the
archive index is only updated once a day.

[1] https://lists.archlinux.org/archives/list/arch-dev-public@lists.archlinux.org/thread/B66SQUW4HOJAN2WHG4RLEBLYWNWOSF3Y/
[2] https://archive.archlinux.org/packages/.all/index.0.xz

Since it's not clear whether that path actually exists it makes sense to
just drop the output as it is not helpful anyways.

Fixes archlinux/dbscripts#51



Signed-off-by: Christian Heusel <christian@heusel.eu>

* upstream/merge-requests/52:
  fix(test): Prevent pacman 6.1 regression with gpg.conf option
  fix(test): Do not use keyboxd as it prevents the use of pacman-key
  fix(db-functions): Add package signatures back to repo desc files
  feat(test): Use rsop instead of sq for OpenPGP operations
  fix(test/Dockerfile): Add debugedit for the test setup

Add a gpg.conf option for pacman's gnupg setup, to prevent a regression
in pacman 6.1 which checks for `lock-never` in gpg.conf and fails when
this option is set.
This is needed because pacman since version 6.1 does no longer set
`lock-never` in its own gnupg configuration.

Signed-off-by: David Runge <dvzrv@archlinux.org>

Create the tester user's ~/.gnupg dir to prevent the default use of
keyboxd, as it disallows using `pacman-key` as an unprivileged user.

Signed-off-by: David Runge <dvzrv@archlinux.org>

With pacman 6.1 the addition of package signatures to the repository
sync database "desc" file has been demoted to the `--include-sigs`
options.

We want signatures in our sync database because it allows us to search
for signer of a package using `pacman -Sii`.

Signed-off-by: David Runge <dvzrv@archlinux.org>

The Stateless OpenPGP (SOP) implementations provide an unchanging CLI,
which is preferable over sq as that breaks on every release.

Use the `.tsk` file ending to indicate that we are dealing with a
'Transferable Secret Key (TSK)' file.

Signed-off-by: David Runge <dvzrv@archlinux.org>

The creation of debug packages requires debugedit.

Signed-off-by: David Runge <dvzrv@archlinux.org>

Signed-off-by: Levente Polyak <anthraxx@archlinux.org>

Signed-off-by: Christian Heusel <christian@heusel.eu>

Calling `acquire_fd` in a subshell doesn't populate the global lockfd
associate array properly. This resulted in the reuse of the same fd
multiple times, effectively only holding a single latest lock and
releasing any previous.

Fix the issue by avoiding a global variable that causes issues in
subshell calls by using the file descriptor table directly. Instead
of storing file descriptors, the lookup call iterates through all fds
and checks their handle. In case a file is not yet opened, allocate the
next free fd between 4 and 1023.

Operating directly on the file descriptor table has the nice side effect
that we avoid reusing descriptors by accident in case they have been
opened for none locking purpose within the statically defined range.

Reported-by: Felix Yan <felixonmars@archlinux.org>
Co-authored-by: Felix Yan <felixonmars@archlinux.org>
Signed-off-by: Levente Polyak <anthraxx@archlinux.org>

Also add explicit logging in case flock fails

This also fixes the workarounds for bats

This avoids the issue of failing `comm` calls due to missing files,
which was previously masked as the script did not have errfail set.

It is mostly never a good idea trying to silently keep going while
errors occur.

In gnupg 2.4 if we do not kill the deamon during container creation it
will leave behind a lock file. This lock leads to hanging gpg processes
when running the container.

Fixes #48

Signed-off-by: Levente Polyak <anthraxx@archlinux.org>

Also lock critical sections to each user has an own lock on their
staging directory to avoid race conditions during multiple processes
handling the same package releases.

Fixes #49
Fixes #38

Signed-off-by: Levente Polyak <anthraxx@archlinux.org>

An initial git clone and especially a git fetch operation can occur
concurrently through multiple packagers and processes. Guard these git
operations with a fetch lock inside the actual git repository as well as
a special lock stored in the shared global lock directory before
starting an initial clone.

Fixes #49

Signed-off-by: Levente Polyak <anthraxx@archlinux.org>

Using read does not work as the previous implementer expected the
feature to work, as a delimiter of d'\n' will not operate like expected.

Use proper line based output parsing by using mapfile to read the output
line by line into an array, and then use array range to reference the
content into variables. This way we can actually check if the subshell
had any errors and abort in case anything went wrong.

Signed-off-by: Levente Polyak <anthraxx@archlinux.org>

This paradigm seems to be in dbscripts since a decade, which effectively
is void. Using a subshell with a local variable declaration basically
always leads to success as the local declaration masks the return code
of the subshell, which means an early exit using `|| return 1` will
never work.

Fix this by splitting local variable declaration from assignment.

Signed-off-by: Levente Polyak <anthraxx@archlinux.org>

Signed-off-by: Levente Polyak <anthraxx@archlinux.org>

Maybe this doesn't matter because the change isn't staged yet anyway,
but it makes me feel safer.

This broke db-update for more than one repo at a time. Most importantly,
because the `pkgkeys` array kept accumulating keys, we created bogus
copies of state repo entries, "releasing" packages into multiple repos.

This gives a more fluent migration phase which does not forcefully
require a coordinated release with devtools.

Fixes #45

Use GIT_TERMINAL_PROMPT=0 to enforce Git not asking for any prompts or
credentials. GitLab does not return none existing repositories but
instead prompts for credentials to avoid side channel information leaks.
However, we explicitly do not want to get any such prompts, instead we
fail gracefully.

This allows to easily override the disabled networking, like passing the
host network to have a simple development environment.