1. 20 Nov, 2021 4 commits
  2. 26 Oct, 2021 1 commit
  3. 05 Jun, 2020 1 commit
    • Eli Schwartz's avatar
      rewrite issplitpkg check to be compatible with pacman 5.1 · eb1a090c
      Eli Schwartz authored
      We tried to optimize out checking packages which aren't even split
      packages, by skipping the ones with no pkgbase. But pacman 5.1 always
      writes the pkgbase in order to simplify parsing of the .PKGINFO, so this
      was no longer reliable. Instead, check how many pkgnames are in the
      corresponding PKGBUILD. This is the master source of truth anyway.
      eb1a090c
  4. 17 Oct, 2019 1 commit
    • Eli Schwartz's avatar
      db-update: performance optimization when checking if a buildinfo exists · 60d036ea
      Eli Schwartz authored
      Use the --fast-read option to quit as soon as the file is first found. A
      package file should never contain two of these anyway, but even if it
      did, we simply want to know that one exists!
      
      makepkg guarantees its internal .METADATA files are tar'ed up first, so
      this should usually be quite rapidly found. For very large packages,
      crawling the entire package file to check for a later entry overriding
      the first one, is... pointless... and worst of all, slow.
      
      This reflects the identical mode used for extracting the pkginfo file,
      which was inconsistently implemented from the initial feature addition.
      60d036ea
  5. 13 Jul, 2019 1 commit
    • Florian Pritz's avatar
      Look for repro packages in live repo pool too · 0706db69
      Florian Pritz authored
      
      
      I've cleaned older packages from the archive, but sometimes we do not
      rebuild packages in a long long time. We still keep have them in the
      repository, but this check does not look for the package there, thus
      when trying to db-update, the user sees an error. We fix this by also
      looking at currently live packages instead of only relying on the
      archive.
      
      This is mostly a hotfix until a better solution is created. Depending on
      when/how the ftpdir-cleanup cronjob removes such packages, users may
      still see errors when an old package is updated and the cronjob removes
      it from the repository before db-update is run.
      
      Signed-off-by: Florian Pritz's avatarFlorian Pritz <bluewind@xinu.at>
      0706db69
  6. 13 Jan, 2019 1 commit
  7. 10 Jan, 2019 1 commit
    • Eli Schwartz's avatar
      db-update: die when trying to update a package without updating a pending rebuild · 8804e63f
      Eli Schwartz authored
      A semi-common pattern is for one maintainer to stage a rebuild of a
      package due to e.g. cascading repository-wide python/boost/whatever
      rebuilds, and then for the original maintainer of the package to not
      notice and update the package in the stable repo, leaving an out of date
      rebuild in staging or testing.
      
      Then the the out of date package gets moved and ends up breaking things,
      possibly via a package downgrade, possibly via breaking compatibility
      with a much more targeted rebuild uploaded all at once. Ultimately,
      Things Happen™ and the repository hierarchy gets broken.
      
      Prevent this by enforcing for all packages that exist in
      multiple levels of the repo: staging -> testing -> stable
      
      That updates to one must come with an update to all the others.
      8804e63f
  8. 09 Jan, 2019 1 commit
    • Eli Schwartz's avatar
      Add reproducible archive of packages. · f11a038c
      Eli Schwartz authored
      Whenever adding new package files to the pool of distributed packages,
      copy the file into a longterm archive. This is the first step to merging
      the functionality of archivetools, as this implements the shared pool
      while also guaranteeing that all packages are archived at the time of
      entry rather than once per day if they still exist.
      f11a038c
  9. 08 Oct, 2018 1 commit
    • Eli Schwartz's avatar
      Preliminary work to break out svn-specific code. · d6b48bd6
      Eli Schwartz authored
      Introduce "db-functions-$VCS" which will eventually contain all
      VCS-specific code, and make this configurable in config.
      
      Move private arch_svn function and svn acl handling here, and introduce
      a new source_pkgbuild function to handle discovering PKGBUILDs from the
      configured VCS and sourcing them to extract metadata.
      
      The PKGBUILD is the only file we ever check out from version control,
      and only ever to scrape information from it, except for when we actually
      want to db-move a whole directory (which is by necessity considerably
      dependent on the VCS in use).
      
      source_pkgbuild is inspired by commits from the dbscripts rewrite,
      authored by Florian Pritz <bluewind@xinu.at>
      d6b48bd6
  10. 09 Sep, 2018 1 commit
    • Levente Polyak's avatar
      fix potential bsdtar stream close error by grep · 0b630e25
      Levente Polyak authored and Eli Schwartz's avatar Eli Schwartz committed
      This silences a useless error message that confuses the user.
      
      bsdtar doesn't like it when the stream gets closed before it finishes
      which may be the case when grep found its match on potentially huge
      archives. Instead of suppressing the whole stderr , we find all matches
      with grep, then use a second pass with `tail` to find only the last
      match, which ensures the stream remains open for bsdtar but we may still
      catch and see useful messages on stderr.
      
      This works because tail has the useful property of not closing early.
      0b630e25
  11. 08 Apr, 2018 1 commit
    • Luke Shumaker's avatar
      Fixup quoting around variables, especially arrays · 5b4b19cd
      Luke Shumaker authored and Eli Schwartz's avatar Eli Schwartz committed
      Search for unquoted variables using the command:
      
          grep -Prn --exclude-dir=.git '(?<!["=]|\[\[ |\[\[ -[zn] )\$(?!{?#|\(|\? )'
      
      and ignore a bunch of false positives.
      
      You may verify that the only differences between the lines removed and
      lines added are double-quotes:
      
              diff -u \
                <(git show|sed -n 's/^-//p'|grep -v '^-- a/'|sed 's/"//g') \
                <(git show|sed -n 's/^+//p'|grep -v '^++ b/'|sed 's/"//g')
      5b4b19cd
  12. 21 Mar, 2018 2 commits
    • Luke Shumaker's avatar
      Clean up printf-formatters for user messages · 8cc8e9cb
      Luke Shumaker authored and Eli Schwartz's avatar Eli Schwartz committed
       - db-functions: getpkgfile: The .sig file extension should be part of
         the filename parameter, not part of the message format string.
      
       - db-functions: arch_repo_modify: Shouldn't use ${action}
         string-interpolation in the message format string.  Since the
         entire message is a command, and we're using @Q to escape arguments
         anyway, go ahead and just construct the entire command as a single
         string that way, and feed it to '%s'.
      8cc8e9cb
    • Luke Shumaker's avatar
      Fixups near unquoted variables · 0432cffc
      Luke Shumaker authored and Eli Schwartz's avatar Eli Schwartz committed
      Using the following command to find unquoted variables (and ignoring
      more than a a few false positives),
      
          grep -Prn --exclude-dir=.git '(?<!["=]|\[\[ |\[\[ -[zn] )\$(?!{?#|\(|\? )'
      
      one is lead to find a few cleanups that are something other than "add
      double-quotes".  That's what these are.  We'll leave dumb adding of
      double-quotes for another commit.
      
      Most of these are still fixing quoting issues, just with a better fix.
      
       - parse_pkgbuilds.sh: Avoid having to escape quotes in `eval` strings
         by using `declare -p`. Updates the logic copied from makepkg, with the
         latest logic copied from makepkg. See
         https://git.archlinux.org/pacman.git/commit/?id=9e52a36794552b77ecf26f7f34b226d096978f1e
       - sourceballs: Avoid using ary=($string) to do field separation by
         using `read` and test that multiple licenses actually work as
         expected.
       - sourceballs: Replace `[[ -z ${ary[*]} ]]` with test for the array
         length
       - db-functions: Replace mangling echo field separators using sed, with
         printf formatters
       - db-functions: Replace for/echo loop to print an array line by line,
         with `printf '%s\n'`
       - db-functions: set_repo_permissions: Line up error messages, quote
         "$group"
       - db-move: Replace `$(echo ${array[@]})` with `${array[*]}`
       - testing2x: Use `"$@"` instead of `$*` when looping over an array
      
      Also, not really quoting related but on the same line as a quoting
      issue, optimize:
      
       - db-functions: Replace
      	[[ -n "$(... | sort | uniq -D)" ]]
         with
      	! ... | awk 'a[$0]++{exit 1}'
      0432cffc
  13. 20 Mar, 2018 4 commits
    • Luke Shumaker's avatar
      Remove uses of the "v=true; if $v ..." anti-pattern · 36087fbd
      Luke Shumaker authored and Eli Schwartz's avatar Eli Schwartz committed
      Instead, compare the value of $v to 'true'.
      36087fbd
    • Luke Shumaker's avatar
      Don't use `grep -q` when operating on piped stdin · 23c2b82c
      Luke Shumaker authored and Eli Schwartz's avatar Eli Schwartz committed
      (By default, prefer `grep &>/dev/null`)
      
      `grep -q` may exit as soon as it finds a match; this is a good optimization
      for when the input is a file.  However, if the input is the output of
      another program, then that other program will receive SIGPIPE, and further
      writes will fail.  When this happens, it might (bsdtar does) print a
      message about a "write error" to stderr.  Which is going to confuse and
      alarm the user.
      
      In one of the cases (in common.bash, in the test suite), this had
      already been mitigated by wrapping bsdtar in "echo "$(bsdtar ...)", as
      Bash builtin echo doesn't complain if it gets SIGPIPE.  However, that
      means we're storing the entire output of bsdtar in memory, which is
      silly.  Additionally, the way it was implemented is also wrong;
      because it was being used with `grep -qv` instead of just `grep -q`,
      it *always* found a non-matching line (even something inconsequential
      like `%NAME%`), and *never* triggered a test failure.
      
      Looking at a few of these cases, it might also make sense to switch to
      using `bsdtar tf` instead of `bsdtar xf` when checking membership, but
      that's work for another day.
      23c2b82c
    • Luke Shumaker's avatar
      Add "#!/hint/bash" to the beginning of several files · 97e17a59
      Luke Shumaker authored and Eli Schwartz's avatar Eli Schwartz committed
      It is a method of notifying text-editors that a file is in Bash syntax
      without giving it a propper shebang (which would be confusing, as it
      would suggest that the file should be executable), as well as working
      across virtually all text-editors (unlike "-*- Mode: Bash -*-" or
      whatever).
      97e17a59
    • Luke Shumaker's avatar
      41333413
  14. 16 Mar, 2018 2 commits
  15. 26 Feb, 2018 1 commit
  16. 22 Feb, 2018 1 commit
    • Eli Schwartz's avatar
      Globally set $PKGEXT to a bash extended glob representing valid choices. · e53cad6e
      Eli Schwartz authored
      The current glob `*.pkg.tar.?z` is both less restrictive and more
      restrictive than makepkg, as it accepts any valid unicode character.
      
      To be more exact, it's almost completely orthogonal to the one in makepkg.
      
      makepkg only accepts .tar.gz, .tar.bz2, .tar.xz, .tar.lzo, .tar.lrz, and
      .tar.Z and most of those fail to match against a two-char compression type.
      
      dbscripts accepts .pkg.tar.💩z which incidentally is what I think of
      cherry-picking xz and gz as supported methods.
      
      Since this can be anything makepkg.conf accepts, it needs to be able to
      match all that, unless we decide to perform additional restrictions in
      which case we should still explicitly list each allowed extension. Using
      bash extended globbing allows us to do this relatively painlessly.
      
      Document the fact that this has *always* been some sort of glob, and
      update the two cases where this was (not!) being evaluated by bash
      [[ ... ]], to use a not-elegant-at-all proxy function is_globfile() to
      evaluate globs *before* testing if they exist.
      e53cad6e
  17. 20 Feb, 2018 2 commits
    • Eli Schwartz's avatar
      db-update: replace external find command with bash globbing · 3e01ba9a
      Eli Schwartz authored
      Don't bother emitting errors. bash doesn't show globbing errors if it
      cannot read a directory to try globbing there. And the former code never
      aborted on errors anyway, as without `set -o pipefail` the sort command
      swallowed the return code.
      3e01ba9a
    • Eli Schwartz's avatar
      Fix overloading PKGEXT to mean two things. · 07e6a91d
      Eli Schwartz authored
      PKGEXT is a makepkg variable referring to a fixed filename suffix, but
      we were also using it to mean a bash glob referring to candidate
      filenames. This is wrong, so rename it to PKGEXTS which is more
      descriptive of its purpose.
      
      Exclude the testsuite from this change, as the testsuite actually uses
      PKGEXT for its intended purpose. Fix the testsuite to consistently use
      PKGEXT, as it hardcoded the file extension in several cases, and pin
      its value to .pkg.tar.xz
      07e6a91d
  18. 15 Feb, 2018 6 commits
    • Eli Schwartz's avatar
      db-functions: deduplicate some repeated logic · 36b71d32
      Eli Schwartz authored
      reuse getpkgfile() in getpkgfiles()
      36b71d32
    • Eli Schwartz's avatar
      Use more bashisms · b61a7148
      Eli Schwartz authored
      Fix numerous instances of POSIX `[ ... ]`, including reliance on ugly
      deprecated constructs like POSIX `-a`. Since we require bash regardless,
      it makes sense to take full advantage of it.
      
      bash `[[ ... ]]` does not require quoting variables as the shell
      natively recognizes them as variables rather than expanded strings.
      
      Use shell arithmetic rather than test, when checking numerical values.
      b61a7148
    • Eli Schwartz's avatar
      Use modern bash to append to strings/arrays. · f4f9d1a0
      Eli Schwartz authored
      Rather than using ugly hacks like arr[${#arr[*]}]="foo", bash 3.1 has
      the += operator.
      
      Update strings to use the same operator while we are at it.
      f4f9d1a0
    • Eli Schwartz's avatar
      ARCHES is an array, do not attempt to call it as a string · a7b497e8
      Eli Schwartz authored
      This meant only the first array element was checked in
      check_repo_permission(). Although arguably this should never cause real
      issues as something else would have to be broken if multiple
      architectures have different permissions, we should catch this now
      anyway.
      a7b497e8
    • Eli Schwartz's avatar
      Update messages to make full use of printf formatters · 5afac1ed
      Eli Schwartz authored
      libmakepkg messaging functions provide automatic access to gettext
      (which we do not currently make use of) in addition to cleanly
      separating data from message strings.
      
      In order for this to work properly, pass argv correctly from die() to
      libmakepkg's error()
      5afac1ed
    • Eli Schwartz's avatar
      Use return codes properly when checking for failed commands. · eec8e35e
      Eli Schwartz authored
      Rather than doing a test(1) on the numerical value of the return code,
      use the `if !` and `||` operators during control flow.
      eec8e35e
  19. 20 Apr, 2017 1 commit
  20. 18 Apr, 2017 3 commits
  21. 16 Feb, 2016 1 commit
  22. 07 Feb, 2016 1 commit
  23. 01 Nov, 2014 1 commit
  24. 02 Nov, 2013 1 commit