archive.sh: Adapt the repo filter for the new repo layout

See merge request !1

related to infrastructure#544


Signed-off-by: Christian Heusel <christian@heusel.eu>

These are not supposed to be archived and dbscripts makes sure not to
add them to /srv/archive/packages/. We just need to exclude them from
our daily repo sync too.

Cleanup sourceballs along with normal packages

When removing a normal package also remove the sourceball package.

Closes: #3

Show estimated cleanup size when running archive-cleaner

 Create index.0 based on all packages in packages dir

This no longer uses only the copy in $PKGFLAT (.all directory), but
instead fetches the names from the subdirs in the packages dir. Right
now this is not an improvement, but it's a preparation so that the .all
directory can be removed and replaced with a redirection in the
webserver.

To prevent an existing .all directory from duplicating the results, the
list is piped through `sort -u`. This can probably be removed again once
the .all directory is removed.

Signed-off-by: Florian Pritz <bluewind@xinu.at>

Default value is cpu count * 2

also change URL to archlinux github

For now it collects referenced packages needed for reproducible
builds by checking .BUILDINFO files of all repo targets.

Later we can use the info to collect artifacts that are free
to delete.

Do not update timed links and packages index when repo_rsync failed.

Mohammad AlSaleh authored 9 years ago and Sébastien Luttringer committed 7 years ago

 pacsort is required to list packages in the right order.

 Example: `agetpkg  -l '^linux$' |tail`

 Before:
 linux 4.3.2-1 x86_64
 linux 4.3.3-1 x86_64
 linux 4.3.3-2 x86_64
 linux 4.3.3-3 x86_64
 linux 4.4.1-1 x86_64
 linux 4.4.1-2 x86_64
 linux 4.4-1 x86_64
 linux 4.4-2 x86_64
 linux 4.4-3 x86_64
 linux 4.4-4 x86_64

 After:
 linux 4.3.2-1 x86_64
 linux 4.3.3-1 x86_64
 linux 4.3.3-2 x86_64
 linux 4.3.3-3 x86_64
 linux 4.4-1 x86_64
 linux 4.4-2 x86_64
 linux 4.4-3 x86_64
 linux 4.4-4 x86_64
 linux 4.4.1-1 x86_64
 linux 4.4.1-2 x86_64

Signed-off-by: Mohammad AlSaleh <CE.Mohammad.AlSaleh@gmail.com>

Reserve the logic of syncing. Now it's a whitelist instead of a blacklist.
This allow us to use kitchensink_tier1 from rsync.al.org to fetch
the source tree, in order to complain with GPLv2 sources distribution.

It may append that rsync stall and don't quit. This add and housekeeper.

- Install in PKGBUILD
- Added in docs
- Get config from /etc/archive.conf