Skip to content
GitLab
  1. 07 Jul, 2020 4 commits
    • Anatol Pomozov's avatar
      FS#33992: force download *.sig file if it does not exist in the cache · f3dfba73
      Anatol Pomozov authored and Allan McRae's avatar Allan McRae committed 
      

In case if *.pkg exists but *.sig file does not we still have to pass
the pkg to multi_download API.

To avoid redownloading *.pkg file we use CURLOPT_TIMECONDITION curl option.

Signed-off-by: Anatol Pomozov's avatarAnatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: Allan McRae's avatarAllan McRae <allan@archlinux.org>
      f3dfba73
    • Anatol Pomozov's avatar
      Add a utility function to check whether a file exists in the cache · 78d6dcec
      Anatol Pomozov authored and Allan McRae's avatar Allan McRae committed 
      

It is similar to _alpm_filecache_find() but does not return a
dynamically allocated memory to user. Thus the user does not need to
free this resource.

Signed-off-by: Anatol Pomozov's avatarAnatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: Allan McRae's avatarAllan McRae <allan@archlinux.org>
      78d6dcec
    • Anatol Pomozov's avatar
      Do not use counter for error tracking · 34ba8d98
      Anatol Pomozov authored and Allan McRae's avatar Allan McRae committed 
      

Current code uses an incrementing counter to check whether a function
returned error:

  errors += some_function();
  if(errors) { goto finish }

Replace with a more standard variable
  errors = some_function();
  if(errors) { goto finish }

Rename 'errors' variable to a more typical 'ret'.

Avoid reporting both ALPM_EVENT_PKG_RETRIEVE_FAILED and
ALPM_EVENT_PKG_RETRIEVE_DONE in the error path.

Signed-off-by: Anatol Pomozov's avatarAnatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: Allan McRae's avatarAllan McRae <allan@archlinux.org>
      34ba8d98
    • Anatol Pomozov's avatar
      Move signature payload creation to download engine · f078c2d3
      Anatol Pomozov authored and Allan McRae's avatar Allan McRae committed 
      Until now callee of ALPM download functionality has been in charge of
payload creation both for the main file (e.g. *.pkg) and for the accompanied
*.sig file. One advantage of such solution is that all payloads are
independent and can be fetched in parallel thus exploiting the maximum
level of download parallelism.

To build *.sig file url we've been using a simple string concatenation:
$requested_url + ".sig". Unfortunately there are cases when it does not
work. For example an archlinux.org "Download From Mirror" link looks like
this https://www.archlinux.org/packages/core/x86_64/bash/download/

 and
it gets redirected to some mirror. But if we append ".sig" to the end of
the link url and try to download it then archlinux.org returns 404 error.

To overcome this issue we need to follow redirects for the main payload
first, find the final url and only then append '.sig' suffix.
This implies 2 things:
 - the signature payload initialization need to be moved to dload.c
 as it is the place where we have access to the resolved url
 - *.sig is downloaded serially with the main payload and this reduces
 level of parallelism

Move *.sig payload creation to dload.c. Once the main payload is fetched
successfully we check if the callee asked to download the accompanied
signature. If yes - create a new payload and add it to mcurl.

*.sig payload does not use server list of the main payload and thus does
not support mirror failover. *.sig file comes from the same server as
the main payload.

Refactor event loop in curl_multi_download_internal() a bit. Instead of
relying on curl_multi_check_finished_download() to return number of new
payloads we simply rerun the loop iteration one more time to check if
there are any active downloads left.

Signed-off-by: Anatol Pomozov's avatarAnatol Pomozov <anatol.pomozov@gmail.com>
Signed-off-by: Allan McRae's avatarAllan McRae <allan@archlinux.org>
      f078c2d3
  3. 26 Jun, 2020 3 commits
    • Denton Liu's avatar
      srcinfo.sh: remove trailing newline · 6b9c1b4d
      Denton Liu authored and Allan McRae's avatar Allan McRae committed 
      

When a .SRCINFO file is generated via `makepkg --printsrcinfo`, each
section is concluded with an empty line. This means that at the end of
the file, an empty line remains. This is considered a trailing
whitespace error. In fact, `git diff --check` will warn about this,
saying "new blank line at EOF."

Instead of closing each section off with an empty line, use the empty
line to separate sections, omitting the empty line at the end of the
file.

Signed-off-by: default avatarDenton Liu <liu.denton@gmail.com>
Signed-off-by: Allan McRae's avatarAllan McRae <allan@archlinux.org>
      6b9c1b4d
    • Anatol Pomozov's avatar
      Cleanup the old sequential download code · 84723cab
      Anatol Pomozov authored and Allan McRae's avatar Allan McRae committed 
      

All users of _alpm_download() have been refactored to the new API.
It is time to remove the old _alpm_download() functionality now.

This change also removes obsolete SIGPIPE signal handler functionality
(this is a leftover from libfetch days).

Signed-off-by: Anatol Pomozov's avatarAnatol Pomozov <anatol.pomozov@gmail.com>
      84723cab
    • Anatol Pomozov's avatar
      Convert '-U pkg1 pkg2' codepath to parallel download · 16d98d65
      Anatol Pomozov authored and Allan McRae's avatar Allan McRae committed 
      

Installing remote packages using its URL is an interesting case for ALPM
API. Unlike package sync ('pacman -S pkg1 pkg2') '-U' does not deal with
server mirror list. Thus _alpm_multi_download() should be able to
handle file download for payloads that either have 'fileurl' field
or pair of fields ('servers' and 'filepath') set.

Signature for alpm_fetch_pkgurl() has changed and it accepts an
output list that is populated with filepaths to fetched packages.

Signed-off-by: Anatol Pomozov's avatarAnatol Pomozov <anatol.pomozov@gmail.com>
      16d98d65
  5. 19 Jun, 2020 1 commit
  7. 15 Jun, 2020 1 commit
  9. 11 Jun, 2020 8 commits
    • Eli Schwartz's avatar
      makepkg/repo-add: do not accept public-only keys for signing · 02ae97b0
      Eli Schwartz authored and Allan McRae's avatar Allan McRae committed 
      

If it's not listed by --list-secret-key we don't care if it has been
imported into your keyring, it's unusable. And you might not have a
private key at all in the no-keyid-specified case.

Signed-off-by: Eli Schwartz's avatarEli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae's avatarAllan McRae <allan@archlinux.org>
      02ae97b0
    • Eli Schwartz's avatar
      makepkg/repo-add: handle GPGKEY with spaces · 899d39b6
      Eli Schwartz authored and Allan McRae's avatar Allan McRae committed 
      

We pass this to gpg -u and this gpg option can accept a number of
different formats, not just the historical hexadecimal fingerprint we
assumed. We should not barf hard if a format is used which happens to
contain spaces.

This also fixes a validation bug. When we initially check if the desired
key is available, we don't quote spaces, so gpg goes ahead and treats
each space-separated string as a *different key* to search for,
returning partial matches, and returning success if at least one key is
found. But gpg --detach-sign -u will certainly not accept multiple keys!

Fixes FS#66949

Signed-off-by: Eli Schwartz's avatarEli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae's avatarAllan McRae <allan@archlinux.org>
      899d39b6
    • Eli Schwartz's avatar
      libmakepkg: fix regression in sending plain() output to stderr · bf458cce
      Eli Schwartz authored and Allan McRae's avatar Allan McRae committed 
      In commit 882e707e we changed message
output to go to stdout by default, unless it was an error. The plain()
function doesn't *look* like an error function, but in practice it was
-- it's used to continue multiline messages, and all in-tree uses were
for warning/error.

This is a problem both because we're sending output to the wrong place,
and because in some cases, we were performing error logging from a
function which would otherwise return a value to be captured in a
variable using command substution.

Fix this and straighten out the API by providing two functions: one for
continuing msg output, and one which wraps this by sending output to
stderr, for continuing error output.

Change all callers to use the second function.
      bf458cce
    • Eli Schwartz's avatar
      makepkg: correctly handle missing download clients · 381e1137
      Eli Schwartz authored and Allan McRae's avatar Allan McRae committed 
      This was broken in commit 882e707e

,
which changed 'plain()' messages to go to stdout, which was then
captured as the download client in question: cmdline=("Aborting...").

The result was a very confusing error message e.g.

/usr/share/makepkg/source/file.sh: line 72: $'\E[1m': command not found

or with makepkg --nocolor:

/usr/share/makepkg/source/file.sh: line 72: Aborting...: command not found

The problem here is that we checked to see if an asynchronous subshell,
in our case <(...), failed, by checking if its captured stdout is
non-empty. Which is terrible, and also a limitation of old bash. But
bash 4.4 can use wait $! to retrieve the return value of an asynchronous
subshell. Now we target that as our minimum, we can sanely handle errors
in such functions.

Losing error messages on stdout by capturing them in a variable instead
of printing them, continues to be a problem, but this will be fixed
systematically in a later commit.

Signed-off-by: Eli Schwartz's avatarEli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae's avatarAllan McRae <allan@archlinux.org>
      381e1137
    • Eli Schwartz's avatar
      makepkg: guard against undefined git pinned sources · 817f9fb7
      Eli Schwartz authored and Allan McRae's avatar Allan McRae committed 
      

If something like source=(..."#commit=") is used, e.g. due to failed
variable expansion, we try to check out an empty refspec as nothing at
all, and end up just running "git checkout". This happens because we
fail at variable expansion too -- so let's quote our variables properly
and make sure git sees this as an empty refspec, so it can error out.

Also make sure it is interpreted as a ref instead of a path.

Signed-off-by: Eli Schwartz's avatarEli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae's avatarAllan McRae <allan@archlinux.org>
      817f9fb7
    • Eli Schwartz's avatar
      build: add libintl dependency to meson and the .pc file · 3bd88821
      Eli Schwartz authored and Allan McRae's avatar Allan McRae committed 
      

In order to use gettext on systems where it is not part of libc, the
correct linker flags are needed in libalpm.pc (for static compilation).
This has never been the case.

The new meson build system currently only checks for ngettext in libc,
but does not fall back to searching for the existence of -lintl; add it
to the libalpm dependencies.

Signed-off-by: Eli Schwartz's avatarEli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Allan McRae's avatarAllan McRae <allan@archlinux.org>
      3bd88821
    • Filipe Laíns's avatar
      ci: cache packages · e348ba38
      Filipe Laíns authored and Allan McRae's avatar Allan McRae committed 
      

Results in ~40s saved in each job.

Signed-off-by: Filipe Laíns's avatarFilipe Laíns <lains@archlinux.org>
      e348ba38
    • Allan McRae's avatar
      Revert "makepkg: drop duplicate reporting of missing dependencies" · 40bbaead
      Allan McRae authored 
      This removed all information on dependency failures if the --syncdeps
flag was not used.  A better approach is needed.

This reverts commit 4246a4cc

.

Signed-off-by: Allan McRae's avatarAllan McRae <allan@archlinux.org>
      40bbaead
  11. 01 Jun, 2020 4 commits
  13. 11 May, 2020 4 commits
  15. 09 May, 2020 13 commits
  17. 06 May, 2020 2 commits