Frédéric Mangano-Tarumi authored Feb 16, 2020 and Lukas Fleischer committed Feb 27, 2020

The new schema was generated with sqlacodegen and then manually adjusted
to fit schema/aur-schema.sql faithfully, both in the organisation of the
code and in the SQL generated by SQLAlchemy.

Initializing the database now requires the new tool aurweb.initdb.
References to aur-schema.sql have been updated and the old schema
dropped.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Add more options to configure the smtplib implementation for sending
notification emails.

The port can be changed using the new smtp-port option.

Encryption can be configured using smtp-use-ssl and smtp-use-starttls.
Keep in mind that you usually also need to change the port when enabling
either of these options.

Authentication can be configured using smtp-user and smtp-password.
Authentication is disabled if either of these values is empty.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Support mail delivery without a local MTA. Instead, an SMTP server can
now be configured using the smtp-server option in the [notifications]
section. In order to use this option, the value of the sendmail option
must be empty.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Frédéric Mangano-Tarumi authored Feb 02, 2020 and Lukas Fleischer committed Feb 02, 2020

First, this gets rid of the deprecation warnings Python displayed.

Second, this fixes the case where a link contained a pair of
underscores, which used to be interpreted as an emphasis because the
linkify processor ran after the emphasis processor.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Frédéric Mangano-Tarumi authored Feb 02, 2020 and Lukas Fleischer committed Feb 02, 2020

When an FS#123 is part of a code block, it must not be converted into a
link. FS#123 may also appear inside an URL, in which case regular
linkifaction of URLs must take precedence.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Frédéric Mangano-Tarumi authored Feb 01, 2020 and Lukas Fleischer committed Feb 02, 2020

Fixes a few edge cases:

- URLs within code blocks used to get redundant <> added, breaking bash
  code snippets like `curl https://...`

 into `curl <https://...>`.

- Links written with markdown's <https://...> syntax also used to get an
  extra pair of brackets.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Frédéric Mangano-Tarumi authored Jan 30, 2020 and Lukas Fleischer committed Feb 02, 2020

Turn the git-commits markdown processor into an inline processor, which
is smart enough not to convert Git hashes contained in code blocks or
links.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Support secondary email addresses that can be used to recover an account
in case access to the primary email address is lost. Reset keys for an
account are always sent to both the primary and the backup email
address.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Since commit eeaa1c3a

 (Separate text from footer in notification emails,
2020-01-04), information about unsubscribing from notifications is added
in a signature block. However, the code to format the email body trimmed
the RFC 3676 signature delimiter, replacing "-- " by "--". Fix this by
adding a special case for signature delimiters.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Stephan Springer authored Jan 04, 2020 and Lukas Fleischer committed Jan 06, 2020

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

We support multiple database backends. Don't require Python modules for
all backends to be installed.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Verify that the update hook exists and is executable before running Git
to prevent from broken repositories when permissions are broken.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Lars Rustand authored Aug 09, 2019 and Lukas Fleischer committed Aug 19, 2019

Will no longer send notifications about "a orphan request", but determine
whether to use a/an based on the first character of the request type.
Signed-off-by: Lars Rustand <rustand.lars@gmail.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

After creating a new account, users need to verify their email address
and set an initial password. Without setting a password, users cannot
use their account on the web interface. However, when logging in via
SSH, we did not check whether the account is verified.

Fix this by only allowing SSH access once a password is set.
Reported-by: Pat Hogan <pathtofile@gmail.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Because filtering by matching the sender && regular expressions on the
subject is awkward.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>

Reverse the order of deletion and addition so that deletion comes first.
This prevents corner cases such as failing unique key constraints when a
provided package changes from lower case to upper case and the old name
is not yet gone.
Helped-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Florian Pritz authored Jan 14, 2019 and Eli Schwartz committed Jan 14, 2019

Signed-off-by: Florian Pritz <bluewind@xinu.at>
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>

Eli Schwartz authored Aug 10, 2018 and Lukas Fleischer committed Aug 12, 2018

In commit f3b4c5c6

 (Refactor the notification script, 2018-05-17), the
parameters of the adopt, disown, comaintainer-add and
comaintainer-remove notification modules were accidentally pushed around
without changing the order in the callers. The notify script now expects
to see the userid followed by additional arguments like the pkgbase id.

As a result, some random userid with the same id as the pkgbase, got
sent a notification regarding some package with the same id as the real
user's id.

Fix this by changing the order in every invocation of the aforementioned
modules.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Add a new configuration option to specify the locale directory to use.
This allows the Python scripts to find the translations, even when not
being run from the source code checkout. At the same time, multiple
parallel aurweb setups can still use different sets of translations.

Fixes FS#59278.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Eli Schwartz authored Jul 08, 2018 and Lukas Fleischer committed Jul 09, 2018

In commit 840ee20f

 (Rename translation resources from aur to aurweb,
2018-07-07) the translations file was renamed but we never actually
switched to using the renamed translations.

As a result, every single push to the AUR contains the following
traceback:

    remote: Traceback (most recent call last):
    remote:   File "/usr/bin/aurweb-notify", line 11, in <module>
    remote:     load_entry_point('aurweb==4.7.0', 'console_scripts', 'aurweb-notify')()
    remote:   File "/usr/lib/python3.6/site-packages/aurweb-4.7.0-py3.6.egg/aurweb/scripts/notify.py", line 541, in main
    remote:   File "/usr/lib/python3.6/site-packages/aurweb-4.7.0-py3.6.egg/aurweb/scripts/notify.py", line 69, in send
    remote:   File "/usr/lib/python3.6/site-packages/aurweb-4.7.0-py3.6.egg/aurweb/scripts/notify.py", line 56, in get_body_fmt
    remote:   File "/usr/lib/python3.6/site-packages/aurweb-4.7.0-py3.6.egg/aurweb/scripts/notify.py", line 192, in get_body
    remote:   File "/usr/lib/python3.6/site-packages/aurweb-4.7.0-py3.6.egg/aurweb/l10n.py", line 14, in translate
    remote:   File "/usr/lib/python3.6/gettext.py", line 514, in translation
    remote:     raise OSError(ENOENT, 'No translation file found for domain', domain)
    remote: FileNotFoundError: [Errno 2] No translation file found for domain: 'aur'
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Mention both the package base name and the request type in the subject
of request closure notification.

Implements FS#41607.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Eli Schwartz authored May 17, 2018 and Lukas Fleischer committed May 18, 2018

Currently we hardcode the architectures the official repos historically
supported, which seems both inefficient because of hardcoding, and
simply wrong, because many packages support various ARM platforms too.

If we were to say "only officially supported arches will be supported in
the AUR" we'd have to disable i686, which seems silly and arbitrarily
restrictive. Also there's better places to implement such a blacklist
(via die_commit in the main loop, via a config option to list supported
arches, would make much more sense in terms of logic).

As for the metadata extraction itself, there's no reason to hardcode the
arches to check for at all. We can get this information too, from the
.SRCINFO itself. Detecting this dynamically is not incompatible with a
blacklist, should we ever decide to implement such a thing.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

User modern Python format() strings with curly braces. Also, convert all
placeholders to named arguments. This allows translators to reorder
messages.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Add support for translating notification emails and send localized
notifications, based on the user's language preferences. Also, update
the translations Makefile to add strings from the notification script
to the message catalog.

Implements FS#31850.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Reimplement most of the notification script logic. Create a separate
class for each notification type. Each class provides methods for
generating the list of recipients, the message subject, the message
body, the references to add at the end of the message and the message
headers. Additionally, a method for sending notification emails is
provided.

One major benefit of the new implementation is that both the generation
of recipients and message contents are much more flexible. For example,
it is now easily possible to make user-specific adjustments to every
single notification of a batch.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Make sure we are consistent with not adding newlines at the end of
notification emails.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Add a script to periodically remove old IP addresses from the users
database.

The login IP addresses are stored for spam protection and to prevent
from abuse. It is quite unlikely that we ever need the IP address of a
user whose last login is more than a week old. It makes sense to remove
such IP addresses to protect our users' privacy.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Eli Schwartz authored Apr 15, 2018 and Lukas Fleischer committed Apr 22, 2018

In the process, rename config.proto to config.defaults (because that is
what it is now).

Also use dict.get('key', default_value) when querying os.environ, rather
than an if block, as it is more pythonic/readable/concise, and reduces
the number of dict lookups.

This change allows aurweb configuration to be done via either:
- copying config.defaults to config and modifying values
- creating a new config only containing modified values, next to a
  config.defaults containing unmodified values

The motivation for this change is to enable ansible configuration in our
flagship deployment by storing only changed values, and deferring to
config.defaults otherwise.

A side benefit is, it is easier to see what has changed by inspecting
only the site configuration file.

If a config.defaults file does not exist next to $AUR_CONFIG or in
$AUR_CONFIG_DEFAULTS, it is ignored and *all* values are expected to
live in the modified config file.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

When sanitizing rendered comments, keep <hr> tags and <br> tags. The
former are generated when using "---" in Markdown comments, the latter
are used when putting two spaces at the end of a line.

Fixes FS#56649.

Johannes Löthberg authored Mar 13, 2018 and Lukas Fleischer committed Mar 13, 2018

Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Johannes Löthberg authored Jan 21, 2018 and Lukas Fleischer committed Jan 21, 2018

git/auth is run as an AutherizedKeysCommand which does not get the
environment variables passed to it, so AUR_OVERWRITE always got
hard-set to '0' by it.  Instead we need to perform the actual privilege
check in git/update instead.
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Automatically detect references to Flyspray bug reports in comments and
convert them to links to the Arch Linux bug tracker.

Implements FS#52008.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Use `/usr/bin/env python3` instead of `/usr/bin/python3` in the shebang
of Python scripts. This adds support for non-standard Python interpreter
paths such as the paths used in virtualenv environments.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Add a missing space to the SQL statement performing the disown
operation.

Fixes FS#55068.

Note that the broken query was not discovered by the test suite since
SQLite parses "?AND" inside prepared statements gracefully while MySQL
does not.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

In addition to the packages list and the package base list, also create
a list of registered user names.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Eli Schwartz authored Jul 24, 2017 and Lukas Fleischer committed Jul 25, 2017

AUR_PRIVILEGED allows people with privileged AUR accounts to evade the
block on non-fast-forward commits. While valid in this case, we should
not do so by default, since in at least one case a TU did this without
realizing there was an existing package.
( https://aur.archlinux.org/packages/rtmidi/

 )

Switch to using allow_overwrite to check for destructive actions.
Use .ssh/config "SendEnv" on the TU's side and and sshd_config
"AcceptEnv" in the AUR server to specifically request overwrite access.
TUs should use: `AUR_OVERWRITE=1 git push --force`
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Eli Schwartz authored Jul 20, 2017 and Lukas Fleischer committed Jul 22, 2017

This was broken in commit 8914a41d

 which
refactored the argument parsing. Instead of checking for at least the
set-keywords command and a pkgbase name, we were checking for *exactly*
the command and pkgbase name, leaving no room for keywords...

As a result, while we could clear the keywords, we could not set them.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>

Since commit 09cb61ab

 (schema: Remove invalid default values for TEXT
columns, 2017-04-15), the PackageBases.FlaggerComment field no longer
has a default value. Initialize this field explicitly whenever a new row
is added to the PackageBases table.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>