This implementation is probably the easiest one and/or cheap. After
building a list of all packages in the repositories we filter the lists
into two separate working lists. One for normal packages and one for
debug packages.

The only issue is that we can encounter globs with no matching cleanup
lists, like when running ftpdir-cleanup after the debug package changes
has been introduced, but before deploying new repositories. We guard
against this by ensuring we have found debug packages, and normal
packages, before checking the working lists.

This introduces some duplicated code but should be fine. Rest of the
logic is mirrored between the two package pools.

Signed-off-by: Morten Linderud <morten@linderud.pw>

Luke Shumaker authored Jun 17, 2014 and Eli Schwartz committed Apr 08, 2018

This does correct handling of
 - executing a program by symlink
 - any weird characters in the full path
 - I'm sure there's another case I thought about when I originally did
   this.

Luke Shumaker authored Mar 15, 2018 and Eli Schwartz committed Apr 08, 2018

Search for unquoted variables using the command:

    grep -Prn --exclude-dir=.git '(?<!["=]|\[\[ |\[\[ -[zn] )\$(?!{?#|\(|\? )'

and ignore a bunch of false positives.

You may verify that the only differences between the lines removed and
lines added are double-quotes:

        diff -u \
          <(git show|sed -n 's/^-//p'|grep -v '^-- a/'|sed 's/"//g') \
          <(git show|sed -n 's/^+//p'|grep -v '^++ b/'|sed 's/"//g')

Luke Shumaker authored Mar 14, 2018 and Eli Schwartz committed Mar 19, 2018

Instead, compare the value of $v to 'true'.

Luke Shumaker authored Feb 22, 2018 and Eli Schwartz committed Mar 15, 2018

These are things that were (IMO) missed in 5afac1ed.  I found them using:

    git grep -E '(plain|msg|msg2|warning|error|die) "[^"]*\$'

I went a little above-and-beyond for escaping strings for the error
messages in db-functions' arch_repo_add and arch_repo_remove.  The
code should explain itself, but I wanted to point it out, as it's more than
the usual "slap %s in there, and move the ${...} to the right".

This fully removes the use of find from the codebase, leads to a
micro-optimization in a couple cases, and ensures that $PKGEXT is
consistently treated as a shell globbing character (which is important
because it is used as one).

Of the eight instances in these files:

- One was unnecessary as `cat` can natively consume all files passed to
  it and no directory traversal was in use.

- Two were unnecessary as they were hardcoded to read a single file....

- Another four were only being used to strip leading directory paths,
  and can be replaced by globstar and ${filepath##*/}

- The final two were checking the modification time of the files, and
  can be replaced with touch(1) and [[ -nt ]]. Although this introduces
  an additional temporary file, this is not such a big deal.

PKGEXT is a makepkg variable referring to a fixed filename suffix, but
we were also using it to mean a bash glob referring to candidate
filenames. This is wrong, so rename it to PKGEXTS which is more
descriptive of its purpose.

Exclude the testsuite from this change, as the testsuite actually uses
PKGEXT for its intended purpose. Fix the testsuite to consistently use
PKGEXT, as it hardcoded the file extension in several cases, and pin
its value to .pkg.tar.xz

Catch some cases that were missed in the previous run.

libmakepkg messaging functions provide automatic access to gettext
(which we do not currently make use of) in addition to cleanly
separating data from message strings.

In order for this to work properly, pass argv correctly from die() to
libmakepkg's error()

Jan Alexander Steffens (heftig) authored Mar 19, 2013 and Pierre Schmitz committed Nov 02, 2013

ACLs from the WORKDIR were used, leading to problems as these weren't
world-readable. Fix follows ftpdir-cleanup.

Signed-off-by: Jan Alexander Steffens (heftig) <jan.steffens@gmail.com>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>

Florian Pritz authored Jan 07, 2013 and Pierre Schmitz committed Jan 20, 2013

Signed-off-by: Florian Pritz <bluewind@xinu.at>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>

Rémy Oudompheng authored Apr 01, 2011 and Pierre Schmitz committed Apr 02, 2011

Signature files are optional and the previous behaviour
is unchanged when signatures files do not exist.

Signed-off-by: Rémy Oudompheng <remy@archlinux.org>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>

This updates the mtime of those files and will help us to remove them later.

Eric Bélanger authored Nov 24, 2010 and Pierre Schmitz committed Nov 25, 2010

Signed-off-by: Eric Bélanger <snowmaniscool@gmail.com>
Signed-off-by: Pierre Schmitz <pierre@archlinux.de>

Using the hostname to decide which repos to use is not releiable and hard to test.
Instead use config.local to configure these.

config files for sigurd and gerolde were added which can be copied or symlinked to config.local
on the specific host.

* runtime reduced to < 1 second
* uses file lists and the comm command
* searches for missing packages
* removes empty legacy directories like extra/os/any

These functions are copied from makepkg

There are no longer architecture-specific subdirs and the structure was switch to this:

ftp
└── pool
    ├── community
    └── packages

packages contains all packages from core, extra and testing; this naming is in sync with the svn repo naming:
svn-packages and svn-community

db-functions now sets an individual $WORKDIR and implements trap functinos that
remove locks on exit or error. There are new functions to lock and unlock the running script.

misc-scripts/ftpdir-cleanup was renamed to ftpdir-cleanup-repo as the cron-job had the same name.
Script names have to be unique when using db-functions.

The package pool dir depends on the host; so we solve this similar to
get_repos_for_host()

Note: There is no "os" subdir for the package pool.

This is a relative var, descendent from FTP_BASE. We do this because
we also need a relative path to the package pool dir

This gets rid of a bash loop for link detection

* The per repo script now simply removes files as they should all be
  symlinks at this time
* The cron job script will do the extra job of checking the global
  pool. If there are no more symlinks, move the pooled package

Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>

And use it. This allows us to have server-specific behavior in our scripts
without further patching, and it also allows us to simplify some of our
scripts a fair amount.

Signed-off-by: Dan McGee <dan@archlinux.org>

This source the config file and gets us functions such as
getpkgname which were duplicated elsewhere

Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>

community repo is cleaned up on a different machine

Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>

This may look like a rather large patch, but the changes are mostly a
reshuffling of the code to loop over all arches first, and then handle
the arch-indep packages. The cronjob has been changed accordingly.
Added new category DELETESYMLINKS, which are deleted instead of being
moved to package-cleanup. I have also fixed the arch-specific issue
with the ftppath, using parameters in config instead.

Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>

These are handled automatically when scanning other architecutres

Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>

Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>

Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com>