GitLab

virtual "can" be used when served under apache2
https://www.php.net/manual/en/function.virtual.php

dl has been removed since php-fpm 7.0.0
https://www.php.net/manual/en/function.dl.php

Disable popen

Disable escapeshellarg

Disable pclose

Disable symlink

Disable shell_exec

Disable proc_open

Disable proc_get_status

Reorder to less annoy jelle:(

Remove symlink from disable_functions

archwiki/includes/media/SvgHandler.php:281:             $ok = symlink( $srcPath, $lnPath );

Remove shell_exec from disable_functions

archwiki/vendor/wikimedia/shellbox/src/Command/UnboxedExecutor.php:173:  * short of shell_exec('env'), but it's usually near-empty anyway. We add
archwiki/maintenance/updateCredits.php:62:$lines = explode( "\n", shell_exec( 'git log --format="%aN"' ) );
archwiki/maintenance/mwdocgen.php:139:          $this->doDot = shell_exec( 'which dot' );
archwiki/extensions/Scribunto/includes/engines/LuaStandalone/LuaStandaloneEngine.php:85:                        self::$clockTick = intval( shell_exec( 'getconf CLK_TCK' ) );

Remove popen from disable_functions

flyspray/plugins/dokuwiki/inc/io.php:516:  $fh = popen($cmd, "r");
archwiki/vendor/wikimedia/parsoid/bin/parse.php:409:                            $pipe = popen( "$fgPath > $fgOutDir/profile.svg", "w" );
archwiki/vendor/pear/pear-core-minimal/src/OS/Guess.php:254:            $cpp = popen("/usr/bin/cpp $tmpfile", "r");
archwiki/maintenance/populateImageSha1.php:117:                 $pipe = popen( $cmd, 'w' );
archwiki/maintenance/includes/SevenZipStream.php:68:            $this->stream = popen( $command, $mode[0] );
archwiki/includes/libs/filebackend/FSFileBackend.php:781:                       $pipes[$index] = popen( $fileOpHandle->cmd, 'r' );
archwiki/includes/GlobalFunctions.php:2202:     $handle = popen( $cmd, 'r' );
archwiki/includes/GlobalFunctions.php:2219:     $handle = popen( $cmd, 'r' );
archwiki/includes/GlobalFunctions.php:2284:     $h = popen( $cmd, 'r' );
archwiki/includes/GlobalFunctions.php:2288:             throw new Exception( __FUNCTION__ . '(): popen() failed' );
archwiki/extensions/Scribunto/includes/engines/LuaStandalone/LuaStandaloneInterpreter.php:196:          $handle = popen( $cmd, 'r' );

Remove pclose from disable_functions

flyspray/plugins/dokuwiki/inc/io.php:522:  pclose($fh);
archwiki/vendor/pear/pear-core-minimal/src/OS/Guess.php:264:            pclose($cpp);
archwiki/vendor/pear/mail/Mail/sendmail.php:184:        $result = pclose($mail);

Remove proc_open from disable_functions

aurweb/web/lib/pkgbasefuncs.inc.php:101:        $p = proc_open($cmd, $descspec, $pipes);
aurweb/web/lib/acctfuncs.inc.php:1334:  $p = proc_open($cmd, $descspec, $pipes);
archwiki/vendor/wikimedia/shellbox/src/Command/UnboxedExecutor.php:223:         $proc = proc_open( $cmd, $desc, $pipes,
archwiki/vendor/wikimedia/parsoid/tools/regression-testing.php:86:              $process = proc_open(
archwiki/vendor/monolog/monolog/src/Monolog/Handler/ProcessHandler.php:118:        $this->process = proc_open($this->command, static::DESCRIPTOR_SPEC, $this->pipes, $this->cwd);
archwiki/tests/parser/editTests.php:293:                $proc = proc_open( '/usr/bin/dwdiff -Pc --diff-input',
archwiki/maintenance/storage/recompressTracked.php:253:                 $proc = proc_open( "$cmd --child-id $i", $spec, $pipes );
archwiki/maintenance/mysql.php:169:             $proc = proc_open( Shell::escape( $args ), $desc, $pipes );
archwiki/maintenance/includes/TextPassDumper.php:793:           $this->spawnProc = proc_open( $cmd, $spec, $pipes );
archwiki/includes/resourceloader/ResourceLoaderImage.php:429:                   $process = proc_open(
archwiki/includes/export/DumpPipeOutput.php:74:         $this->procOpenResource = proc_open( $command, $spec, $pipes );
archwiki/includes/exception/ShellDisabledError.php:35:          parent::__construct( 'Unable to run external programs, proc_open() is disabled' );
archwiki/includes/GlobalFunctions.php:2076:             return 'Unable to run external programs, proc_open() is disabled.';
archwiki/extensions/Scribunto/includes/engines/LuaStandalone/LuaStandaloneInterpreter.php:147:          $this->proc = proc_open(

Remove proc_get_status from disable_functions

archwiki/vendor/wikimedia/shellbox/src/Command/UnboxedExecutor.php:270:                         $status = proc_get_status( $proc );
archwiki/vendor/wikimedia/shellbox/src/Command/UnboxedExecutor.php:358:                 $status = proc_get_status( $proc );
archwiki/extensions/Scribunto/includes/engines/LuaStandalone/LuaStandaloneInterpreter.php:626:                  $status = proc_get_status( $this->proc );

Remove escapeshellarg from disable_functions

flyspray/includes/class.flyspray.php:1477:               $type = @exec(sprintf('file -bi %s', escapeshellarg($fname)));
aurweb/web/lib/acctfuncs.inc.php:1247:  $cmd = "/usr/bin/ssh-keygen -l -f " . escapeshellarg($tmpfile);
aurweb/web/lib/acctfuncs.inc.php:1326:          $cmd .= ' ' . escapeshellarg($param);
archwiki/vendor/wikimedia/shellbox/src/Shellbox.php:148:                                $retVal .= escapeshellarg( $arg );
archwiki/vendor/wikimedia/shellbox/src/Command/BashWrapper.php:32:                      $cmd = '/bin/bash ' . escapeshellarg( __DIR__ . '/limit.sh' ) . ' ' .
archwiki/vendor/wikimedia/shellbox/src/Command/BashWrapper.php:37:                                      'SB_CGROUP=' . escapeshellarg( $this->cgroup ) . '; ' .
archwiki/vendor/pear/pear-core-minimal/src/System.php:81:                $escape = escapeshellarg($b);
archwiki/vendor/pear/mail/Mail/sendmail.php:172:        $from = escapeshellarg($from); // Security bug #16200
archwiki/includes/libs/filebackend/FSFileBackend.php:825:               $encSrc = escapeshellarg( $this->cleanPathSlashes( $fsSrcPath ) );
archwiki/includes/libs/filebackend/FSFileBackend.php:826:               $encStage = escapeshellarg( $this->cleanPathSlashes( $fsStagePath ) );
archwiki/includes/libs/filebackend/FSFileBackend.php:827:               $encDst = escapeshellarg( $this->cleanPathSlashes( $fsDstPath ) );
archwiki/includes/libs/filebackend/FSFileBackend.php:857:               $encSrc = escapeshellarg( $this->cleanPathSlashes( $fsSrcPath ) );
archwiki/includes/libs/filebackend/FSFileBackend.php:858:               $encDst = escapeshellarg( $this->cleanPathSlashes( $fsDstPath ) );
archwiki/includes/libs/filebackend/FSFileBackend.php:878:               $encSrc = escapeshellarg( $this->cleanPathSlashes( $fsPath ) );

Address jelle's comments

Otherwise running terraform under tf-stage2 will often fail with:

> ansible.errors.AnsibleError: Vault password client script
> ../misc/vault-keyring-client.sh did not find a secret for
> vault-id=default: b'gpg: decryption failed: No secret key\n'

gitlab-exporter: add gitlab-exporter to monitoring

See merge request archlinux/infrastructure!566

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>

Bash histories indicate this isn't being used anywhere other than
{build,gemini}.archlinux.org and gemini's filelist is so big that
locate becomes so slow that it's practically useless on this box.

Onboard artafinde as Junior DevOps

Closes #452

See merge request archlinux/infrastructure!567

artafinde is our new newest Junior DevOp[1] and will get access to:
* monitoring.al.org: for setting up gitlab-exporter[1]
* gitlab.al.org: for setting up gitlab-exporter[1]
* dashboards.al.org: in case he wants to do more monitoring related
  stuff

[1] https://lists.archlinux.org/pipermail/arch-devops/2022-May/000558.html
[2] https://gitlab.archlinux.org/artafinde/gitlab-exporter/

Fix #452

Move highly sensitive secrets to new "super" vault

See merge request archlinux/infrastructure!565

- group_vars/all/vault_mariadb.yml: remove 'zabbix' database user
- misc/vaults/additional-credentials.vault: remove zabbix irc bot
- roles/dbscripts/tasks/main.yml: drop unused tier0 mirror access

The idea bebind this is to be able to give vault access to new DevOps
members without giving away more important credentials like Hetzner's.

These were previously removed temporarily and re-created several minutes
later during the process of deploying archusers to gemini.archlinux.org.

Add additional pubkey for dvzrv

See merge request archlinux/infrastructure!568

pubkeys/dvzrv.pub:
Add pubkey based on auth subkey of PGP key
`1793DAD5D803A8FFD7451697BB992F9864FAD168`.

geomirror: leverage LUA records for failover+GeoIP

See merge request archlinux/infrastructure!563

In an effort to stay consistent with the TTL used for the archlinux.org
and pkgbuild.com NS records, as well as slightly improve lookup latency.

PowerDNS provides a neat way to implement GeoIP-based redirection and
automatic failover. With GeoLite2-City database, it is able to select
the closest mirror from a list of IPs we provide. Every 60 seconds it
also checks if the mirror's HTTPS URL is working as expected; if that
check fails, it stops giving it out (this acts as automatic failover).

archbuild: Distribute CPU and IO resources equally among users

See merge request archlinux/infrastructure!564

archbuild: Turn off Git's safe.directory

See merge request archlinux/infrastructure!561

Without this setting, Git exits with an error when the repository is not
owned by the current user. This messes with our shared srcdest.

Packer bootstrap tweaks

See merge request archlinux/infrastructure!562

New hcloud adds protection fields to servers, volumes and floating IPs.

Since we are now using the local disk instead of a volume (which can be
scaled up easily) it helps to have a more consistent view of free space.

All database user passwords have been updated to use scram-sha-256, so
there's no need for backward compatibility with md5.

Also remove the suggestion to call delete_old_cluster.sh; it's now being
created under /tmp and it only contains a command to remove the old data
directory. (We can do the latter ourselves after some time has passed.)

Ensure the correct version is installed and matches $FROM_VERSION.