Skip to content
Snippets Groups Projects

2020 05 06

Last edited by Christian Heusel

Gitlab Proof of Concept

The current Gitlab PoC runs on a Hetzner CX22 (2 cores, 8GB ram, 80 GB disk space) with two Gitlab runner on runner1.archlinux.org which is a PIA box with 8 CPU's, 8 GB's of ram and an HDD and a runner2.archlinux.org on a packet.net server.

GitLab <-> GitHub sync

As some of our projects are on Github it would make sense to mirror these projects (or all official Arch Linux projects) and allow Github users to make Pull requests which have to be merged on Github and then synced to Gitlab. The downside would be the lack of Gitlab CI on the Github

Two options to sync:

https://docs.gitlab.com/ee/user/project/repository/repository_mirroring.html https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/github_integration.html#using-gitlab-cicd-with-a-github-repository-premium

Additionally or as alternative Github users could be allowed to log in on our Keycloak instance via OAUTH 2 https://docs.gitlab.com/ee/integration/github.html

The advantage of using a service account on GitHub with a personal access token is that we also get CI on the GitHub side.

Tasks

GitLab security considerations

  • Are there any measures we need to take to improve the current Gitlab security?
  • Get our Keycloak and GitLab configs audited by some experienced folks
    • freswa: TBD
    • jleclanche: TBD
    • anthrax: TBD
    • Sam: TBD

Security settings

  • Gitlab allows restricting allowed ssh key types (DSA, RSA) or enforce a certain sized key (1024 bits).
  • Personal token lifetime
  • Storage space restrictions
  • Max password size for signup (keycloak?? 256 alphanumeric password does not seem to be an issue)
  • Abuse reports go to (abuse@archlinux.org) which does not exist
  • IP Rate limits (https://gitlab.archlinux.org/admin/application_settings/network)
  • Protected web paths.
  • Do we allow user pages?
  • Only allow pipelines for Arch Projects?
  • Limit pipeline duration?

https://gitlab.archlinux.org/admin/application_settings/general

Runner configuration

  • Do we want a special runner to allow for building/testing VMs and ISOs and such things requiring privileged access?
    • Suggestion: Use runner A with /dev/kvm access for CI for untrusted testing (such as PRs) and runner B with /dev/kvm for building a certain trusted/protected branch only. This way we keep testing untrusted but still allow for checking PRs while we can trust the build automation.
  • Who is allowed to use that proposed Runner?
  • What are the requirements of such a Runner and which projects need it?

GitLab general user collaboration

  • What features do we enable for our users?
  • How do we monitor abuse of personal repository or forks? (excessive data usage for example)
  • Set account limits on push size, attachment size, and max. namespace storage size
  • Onboarding of users requirements, do we require a ToS / privacy policy (GDPR comes to mind)
  • Spam/bot signup prevention, Gitlab offers Recaptcha/Akismet. https://gitlab.archlinux.org/admin/application_settings/reporting
  • It's possible to set up default limits to, for instance, 1GiB (which is the limit users are bound to) and then overwrite this limit for our own groups

Gitlab groups/roles?

On Github we have groups of "maintainers" do we replicate this? On luna we also have groups:

archiso-git:x:1070:dan,aaron,thomas,pierre,dvzrv
devtools-git:x:1072:aaron,dan,thomas,allan,pierre,dreisner,foutrelis,svenstaro,heftig,bpiotrowski,seblu,anthraxx,jelle
initscripts-git:x:1073:thomas,aaron,allan,dan,tomegun
mkinitcpio-git:x:1074:aaron,thomas,tpowa,dreisner,foutrelis,grazzolini
namcap-git:x:1075:aaron,allan,dan,remy,kkeen,svenstaro
pacbuild-git:x:1076:
pacman-git:x:1077:dan,aaron,allan,dreisner,andrew
klibc-extras-git:x:1078:thomas,aaron
archboot-git:x:1079:tpowa
srcpac-git:x:1080:dan,allan,andrea
installer-git:x:1081:tpowa,dan,aaron
netcfg-git:x:1082:andrea,thomas,remy,bluewind,bpiotrowski
dbscripts-git:x:1083:aaron,dan,thomas,pierre,svenstaro,bpiotrowski,eschwartz,bluewind
abs-git:x:1084:dan,allan,dreisner
archstats-git:x:1085:
archweb-git:x:1086:aaron,dan,pierre,thomas,foutrelis,bluewind,jelle
linux-git:x:1087:thomas,tpowa,heftig,anthraxx
newsletter-git:x:1088:
server-misc-git:x:1089:
aif-git:x:1090:thomas
vhosts-wiki-git:x:1091:pierre,thomas,bluewind,bpiotrowski
vhosts-bbs-git:x:1092:andrea,dan,pierre,thomas,bluewind
vhosts-bugs-git:x:1093:jgc,pierre,dan,foutrelis,grazzolini,bluewind,jelle
vhosts-repos-git:x:1094:thomas
kde-build-git:x:1095:pierre,andrea,svenstaro,felixonmars,arojas
ais-git:x:1096:dreisner,eschwartz
netctl-git:x:1097:jouke
vhosts-projects-git:x:1098:lfleischer,seblu,bluewind
planet-git:x:1099:andrea,bpiotrowski,svenstaro,bluewind,jelle
infrastructure-git:x:1100:bluewind,svenstaro,seblu,bpiotrowski,heftig,grazzolini,foutrelis,jelle,fukawi2,pierre,anthraxx
pacman-contrib-git:x:1117:allan,demize,polyzen
pyalpm-git:x:1120:jelle
archlinux-keyring-git:x:1121:eworm,bpiotrowski,jelle,pierre,bluewind,grazzolini
aurweb-git:x:1071:eschwartz,lfleischer
cgit-git:x:1126:lfleischer,eworm

Gitlab backups

Setup our normal borg role, possible having to add /var/lib/docker to the excludes?

What Gitlab runners to use and on what hardware?

Currently we have two runners:

  • runner1.archlinux.org - PIA box (4 cores, 8GB ram, HDD)
  • runner2.archlinux.org - packet.net box

Do we apply restrictions on runners and projects?

Monitoring Gitlab

Currently, nothing is integrated into our monitoring for Gitlab, how does a normal admin monitor Gitlab?

An option is to hook up a Prometheus to our Grafana instance.

https://docs.gitlab.com/ee/administration/monitoring/prometheus/

what goes onto GitLab?

On our GitLab, we host our official Arch Linux projects

git.archlinux.org migrations

  • can we abolish git.archlinux.org? (reduce maintenance)
    • Most likely needs to be kept for svn2git and the AUR?
  • do we archive old projects?
    • Obsolete or Dead Projects
    • Developer projects
  • do we redirect old projects to the new location?

How do we migrate Bugs?

  • What is the layout of our future bug tracker?
    • Do we migrate Arch and/or AUR Packages?
    • One bug tracker/project for all community/core/extra packages?
    • One bug tracker/project per package?
      • If we do not migrate the packages, we need to ensure consistency across bug tracker and package database
  • What are the possible ways for a migration?
    • Includes bugs of projects, Pacman, core/extra, community, AUR, Release Engineering, ISO
    • Do a complete import of Bugzilla while migrating also all the junk we have and loosing some data such as the created date
    • Do not migrate anything and keep Flyspray running in RO mode, where no new Issues are allowed and should end up in the GitLab
    • Do a bug day first where we close as many bugs as possible and then do a migration afterwards within a given timeframe after which Flyspray is shut off
  • Bug day (in case we do it)
    • We need guidelines about how we handle (very) old bugs
    • Who writes a proposal until when?
  • Migration script (in case we use it)
    • Do we migrate single bugs or whole lists?
    • Who writes a script until when?

How do we allow access to Keycloak?

  • Force 2FA? For everyone? Just staff? Just DevOps? Anyone?
  • Allow brokering? (Allows for social logins via Gitlab.com, Github.com, Google to our Keycloak instance)

what goes onto Keycloak?

Do we limit SSO Migration

Next projects to migrate to Keycloak?

The easiest projects to onboard on Keycloak would be our internal infrastructure without such as:

  • Grafana
  • Archweb

SSO Migration

Comments

    Please register or sign in to add a comment.