Johannes Löthberg authored Oct 01, 2021 and Jelle van der Waa committed Oct 17, 2021

Otherwise if you use the dbscripts for core/extra for community repos,
or vice versa, you'll get an error saying that you don't have
permissions to move packages from or to the given repositories, which
can be confusing if you don't immediately notice your mistake.

Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>

Used with GitLab's test coverage visualization[1] and test coverage
results[2].

[1] https://docs.gitlab.com/ee/user/project/merge_requests/test_coverage_visualization.html
[2] https://docs.gitlab.com/ee/ci/pipelines/settings.html#add-test-coverage-results-to-a-merge-request

travisci is no longer used.

This makes it possible to run the tests with podman which has the
benefit of requiring no daemon.

Morten Linderud authored Apr 27, 2021 and Jelle van der Waa committed Oct 17, 2021

We are currently deploying a bunch of AGPL3 software in our devops team
and all the source for the should be available to meet license
requirements.

This is a "hack" so we can access the sources from our Arch Linux
ftpdir.

Signed-off-by: Morten Linderud <morten@linderud.pw>

Luke Shumaker authored Apr 03, 2021 and Jelle van der Waa committed Oct 17, 2021

I hereby confirm that I license my work on dbscripts under the
GPLv2-or-later.

Kristian Klausen authored Jun 23, 2021 and Eli Schwartz committed Jun 23, 2021

All the arch-x@archlinux.org -> arch-x@lists.archlinux.org aliases will
be dropped soon[1].

[1] https://lists.archlinux.org/pipermail/arch-dev-public/2021-June/030462.html



Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>

The official image switched names from "base" to "archlinux".

BUILDDIR is a testsuite requirement, cases might be set manually

Or else it doesn't work if you have packages installed but not from a repo

We tried to optimize out checking packages which aren't even split
packages, by skipping the ones with no pkgbase. But pacman 5.1 always
writes the pkgbase in order to simplify parsing of the .PKGINFO, so this
was no longer reliable. Instead, check how many pkgnames are in the
corresponding PKGBUILD. This is the master source of truth anyway.

Further contributions will need to be consistent with this, and now we
need to hunt down the remaining contributors and clarify the license
status. Fun!

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>

This adds support for zst, lz4, and lzip compressed packages.

Use the --fast-read option to quit as soon as the file is first found. A
package file should never contain two of these anyway, but even if it
did, we simply want to know that one exists!

makepkg guarantees its internal .METADATA files are tar'ed up first, so
this should usually be quite rapidly found. For very large packages,
crawling the entire package file to check for a later entry overriding
the first one, is... pointless... and worst of all, slow.

This reflects the identical mode used for extracting the pkginfo file,
which was inconsistently implemented from the initial feature addition.

I've cleaned older packages from the archive, but sometimes we do not
rebuild packages in a long long time. We still keep have them in the
repository, but this check does not look for the package there, thus
when trying to db-update, the user sees an error. We fix this by also
looking at currently live packages instead of only relying on the
archive.

This is mostly a hotfix until a better solution is created. Depending on
when/how the ftpdir-cleanup cronjob removes such packages, users may
still see errors when an old package is updated and the cronjob removes
it from the repository before db-update is run.

Signed-off-by: Florian Pritz <bluewind@xinu.at>

This needs to check the pkgbase, not the pkgname, as for split packages
we need to lookup a PKGBUILD using the basename.

Not sure what I thought here.

Don't allow anomalous testing packages floating around after a rebuild
which are older than stable.

A semi-common pattern is for one maintainer to stage a rebuild of a
package due to e.g. cascading repository-wide python/boost/whatever
rebuilds, and then for the original maintainer of the package to not
notice and update the package in the stable repo, leaving an out of date
rebuild in staging or testing.

Then the the out of date package gets moved and ends up breaking things,
possibly via a package downgrade, possibly via breaking compatibility
with a much more targeted rebuild uploaded all at once. Ultimately,
Things Happen™ and the repository hierarchy gets broken.

Prevent this by enforcing for all packages that exist in
multiple levels of the repo: staging -> testing -> stable

That updates to one must come with an update to all the others.

Currently multilib is a second-class citizen the way it is lumped into
community, and dbscripts cannot even keep track of whether it
constitutes a testing repo. Teach config to track both testing and
staging repos just like the stable ones, and teach testing2x to search
through TESTING_REPOS to determine which one it is operating on.

At the point when it was being run, the signature was not yet moved to
PKGPOOL.

Whenever adding new package files to the pool of distributed packages,
copy the file into a longterm archive. This is the first step to merging
the functionality of archivetools, as this implements the shared pool
while also guaranteeing that all packages are archived at the time of
entry rather than once per day if they still exist.

Prerequisite for reproducible archives of packages.

Luke Shumaker authored Jul 15, 2018 and Eli Schwartz committed Oct 16, 2018

 - Implement the TODO by keeping a list/set of architectures found via
   "$repo-$arch" directory names, and another list/set of architectures
   named in arch=() in the PKGBUILD(s).  This means turning the simple
   `compgen` in to a loop.

 - While we're at it loading PKGBUILDs in a loop, fix that it clearly
   isn't doing anything with the $pkgver argument; it should verify that
   the version in the found PKGBUILD(s) matches that argument.

 - Use extglob to more strictly match the "arch" part of the "repo-arch"
   dirname; the existing glob wouldn't have behaved correctly for values
   of $repo containing a "-" (like "community-testing").  We don't
   currently test with any of those, but it makes me nervous.

 - Also make that extglob change in checkRemovedPackage.  While we're at
   it, let Bash do the glob expansion normally and check it with
   __isGlobfile, instead of compgen; it means we don't have to worry about
   escaping the non-glob part of the string.  Not that we expect it to
   contain anything needing escaping, but again, it makes me nervous.

Luke Shumaker authored Jul 15, 2018 and Eli Schwartz committed Oct 16, 2018

It is important that db-remove be able to remove a single pkgname, without
being able to look it up by pkgbase in SVN.  For instance, when a split
package update removes one of its members; there will be no reference to
the removed pkgname in SVN, and it won't be removed by db-update.  If
db-remove doesn't accept pkgnames, then this outdated orphan could not be
removed.

As of the source_pkgbuild rewrite, this is only ever done once.

Introduce "db-functions-$VCS" which will eventually contain all
VCS-specific code, and make this configurable in config.

Move private arch_svn function and svn acl handling here, and introduce
a new source_pkgbuild function to handle discovering PKGBUILDs from the
configured VCS and sourcing them to extract metadata.

The PKGBUILD is the only file we ever check out from version control,
and only ever to scrape information from it, except for when we actually
want to db-move a whole directory (which is by necessity considerably
dependent on the VCS in use).

source_pkgbuild is inspired by commits from the dbscripts rewrite,
authored by Florian Pritz <bluewind@xinu.at>

Luke Shumaker authored Jul 13, 2018 and Eli Schwartz committed Oct 07, 2018

The problem statement:

  checkPackageDB and checkRemovedPackageDB need bit of information on the
  package they're checking: a full list of pkgnames with that pkgbase, the
  list of pkgarches, and (for checkPackageDB only) the full version.  In
  dbscripts itself, we can get that directly from the .db files; however,
  since the test suite is checking the ability of dbscripts to edit those
  .db files, that's obviously not a good solution.

  The current solution is to get this information from the PKGBUILD...
  which we also count on dbscripts to correctly keep track of.  Wait,
  that's skipping ahead, let's back up.

  The current solution is to get this information from the PKGBUILD.  For
  checkRemovedPackageDB, that's easy; we just get it from trunk, as that's
  the most up-to-date information on the package as-it-would-have-existed
  (if that sounded a little hand-wavey, it was).  But for checkPackageDB,
  it's a little trickier, because of 2 factors working together: (1) there
  might be different versions on different repos, and (2) unlike
  checkRemovedPackageDB, checkPackageDB actually cares about pkgver.  So,
  checkPackageDB "guesses" the location in a slightly sloppy way, and is
  tagged with a "FIXME".

What todo about it?

  There are a couple of things to observe:

   - Hidden in the hand-waving in assuming that "trunk" is fine for
     checkRemovedPackageDB is the assumption that neither pkgname=() nor
     arch=() is going to change between versions.  Which is a fine
     assumption, because we don't change those things between versions in
     any of our test cases.
   - We're counting on dbscripts correctly keeping track of which PKGBUILD
     is correct for which repo... which is one of the thing's we're trying
     to test, we shouldn't be counting on it.  That's actually a bigger
     problem than the original "FIXME"!

  So, putting those things together, let's (1) take the code under test out
  of the equation, and (2) remove any suggestion that the version of the
  PKGBUILD makes a difference to pkgnames/pkgarches: Let's have both
  functions that that information from the PKGBUILDs under "fixtures/",
  rather than getting PKGBUILDs from VCS.

  That just leaves one question: How to get the correct pkgver in
  checkPackageDB?  The obvious answer is: Each test case knows what the
  version should be; add it as an argument, and adjust every testcase that
  calls it.

Luke Shumaker authored Jul 13, 2018 and Eli Schwartz committed Oct 07, 2018

With BATS setting up error traps, we get used to writing simple `[[ foo ]]`
assertions and not having to check the result.  However, using `!` to
invert the result DOESN'T trigger the trap.  It works OK when it is the
last command in a function, as it still affects the function's return
value, so then the trap triggers in the caller (rather than in the
function).  This means that a check may run successfully when it should
fail.

So, replace all uses of bare `! cmd` with `if cmd; then return 1; fi`,
unless it is the final statement in a function (as it is in
sourceballs.bats:__checkRemovedSourcePackage())

The mistake meant that checkRemovedPackage() was effectively equivalent to
checkRemovedPackageDB().  This meant that no one noticed that db-updates's
"add package with inconsistent name fails" test called checkRemovedPackage
instead of checkRemovedPackageDB.  Fixing the ! issue means that the test
now fails, so change which function it calls.

Levente Polyak authored Sep 03, 2018 and Eli Schwartz committed Sep 09, 2018

This silences a useless error message that confuses the user.

bsdtar doesn't like it when the stream gets closed before it finishes
which may be the case when grep found its match on potentially huge
archives. Instead of suppressing the whole stderr , we find all matches
with grep, then use a second pass with `tail` to find only the last
match, which ensures the stream remains open for bsdtar but we may still
catch and see useful messages on stderr.

This works because tail has the useful property of not closing early.

Levente Polyak authored Sep 03, 2018 and Eli Schwartz committed Sep 09, 2018

The old travis-ci.org is deprecated and this project was migrated.