Gitlab Proof of Concept
The current Gitlab PoC runs on a Hetzner CX22 (2 cores, 8GB ram, 80 GB disk space) with two Gitlab runner on runner1.archlinux.org which is a PIA box with 8 CPU's, 8 GB's of ram and an HDD and a runner2.archlinux.org on a packet.net server.
GitLab <-> GitHub sync
As some of our projects are on Github it would make sense to mirror these projects (or all official Arch Linux projects) and allow Github users to make Pull requests which have to be merged on Github and then synced to Gitlab. The downside would be the lack of Gitlab CI on the Github
Two options to sync:
- GitHub personal access token https://docs.gitlab.com/ee/user/project/repository/repository_mirroring.html#setting-up-a-push-mirror-from-gitlab-to-github-core
- SSH keys
Additionally or as alternative Github users could be allowed to log in on our Keycloak instance via OAUTH 2 https://docs.gitlab.com/ee/integration/github.html
The advantage of using a service account on GitHub with a personal access token is that we also get CI on the GitHub side.
GitLab security considerations
- Are there any measures we need to take to improve the current Gitlab security?
- Get our Keycloak and GitLab configs audited by some experienced folks
- freswa: TBD
- jleclanche: TBD
- anthrax: TBD
- Sam: TBD
- Gitlab allows restricting allowed ssh key types (DSA, RSA) or enforce a certain sized key (1024 bits).
- Personal token lifetime
- Storage space restrictions
- Max password size for signup (keycloak?? 256 alphanumeric password does not seem to be an issue)
- Abuse reports go to (firstname.lastname@example.org) which does not exist
- IP Rate limits (https://gitlab.archlinux.org/admin/application_settings/network)
- Protected web paths.
- Do we allow user pages?
- Only allow pipelines for Arch Projects?
- Limit pipeline duration?
- Do we want a special runner to allow for building/testing VMs and ISOs and such things requiring privileged access?
- Suggestion: Use runner A with /dev/kvm access for CI for untrusted testing (such as PRs) and runner B with /dev/kvm for building a certain trusted/protected branch only. This way we keep testing untrusted but still allow for checking PRs while we can trust the build automation.
- Who is allowed to use that proposed Runner?
- What are the requirements of such a Runner and which projects need it?
GitLab general user collaboration
- What features do we enable for our users?
- How do we monitor abuse of personal repository or forks? (excessive data usage for example)
- Set account limits on push size, attachment size, and max. namespace storage size
- Spam/bot signup prevention, Gitlab offers Recaptcha/Akismet. https://gitlab.archlinux.org/admin/application_settings/reporting
- It's possible to set up default limits to, for instance, 1GiB (which is the limit users are bound to) and then overwrite this limit for our own groups
On Github we have groups of "maintainers" do we replicate this? On luna we also have groups:
archiso-git:x:1070:dan,aaron,thomas,pierre,dvzrv devtools-git:x:1072:aaron,dan,thomas,allan,pierre,dreisner,foutrelis,svenstaro,heftig,bpiotrowski,seblu,anthraxx,jelle initscripts-git:x:1073:thomas,aaron,allan,dan,tomegun mkinitcpio-git:x:1074:aaron,thomas,tpowa,dreisner,foutrelis,grazzolini namcap-git:x:1075:aaron,allan,dan,remy,kkeen,svenstaro pacbuild-git:x:1076: pacman-git:x:1077:dan,aaron,allan,dreisner,andrew klibc-extras-git:x:1078:thomas,aaron archboot-git:x:1079:tpowa srcpac-git:x:1080:dan,allan,andrea installer-git:x:1081:tpowa,dan,aaron netcfg-git:x:1082:andrea,thomas,remy,bluewind,bpiotrowski dbscripts-git:x:1083:aaron,dan,thomas,pierre,svenstaro,bpiotrowski,eschwartz,bluewind abs-git:x:1084:dan,allan,dreisner archstats-git:x:1085: archweb-git:x:1086:aaron,dan,pierre,thomas,foutrelis,bluewind,jelle linux-git:x:1087:thomas,tpowa,heftig,anthraxx newsletter-git:x:1088: server-misc-git:x:1089: aif-git:x:1090:thomas vhosts-wiki-git:x:1091:pierre,thomas,bluewind,bpiotrowski vhosts-bbs-git:x:1092:andrea,dan,pierre,thomas,bluewind vhosts-bugs-git:x:1093:jgc,pierre,dan,foutrelis,grazzolini,bluewind,jelle vhosts-repos-git:x:1094:thomas kde-build-git:x:1095:pierre,andrea,svenstaro,felixonmars,arojas ais-git:x:1096:dreisner,eschwartz netctl-git:x:1097:jouke vhosts-projects-git:x:1098:lfleischer,seblu,bluewind planet-git:x:1099:andrea,bpiotrowski,svenstaro,bluewind,jelle infrastructure-git:x:1100:bluewind,svenstaro,seblu,bpiotrowski,heftig,grazzolini,foutrelis,jelle,fukawi2,pierre,anthraxx pacman-contrib-git:x:1117:allan,demize,polyzen pyalpm-git:x:1120:jelle archlinux-keyring-git:x:1121:eworm,bpiotrowski,jelle,pierre,bluewind,grazzolini aurweb-git:x:1071:eschwartz,lfleischer cgit-git:x:1126:lfleischer,eworm
Setup our normal borg role, possible having to add /var/lib/docker to the excludes?
What Gitlab runners to use and on what hardware?
Currently we have two runners:
- runner1.archlinux.org - PIA box (4 cores, 8GB ram, HDD)
- runner2.archlinux.org - packet.net box
Do we apply restrictions on runners and projects?
Currently, nothing is integrated into our monitoring for Gitlab, how does a normal admin monitor Gitlab?
An option is to hook up a Prometheus to our Grafana instance.
what goes onto GitLab?
On our GitLab, we host our official Arch Linux projects
- can we abolish git.archlinux.org? (reduce maintenance)
- Most likely needs to be kept for svn2git and the AUR?
- do we archive old projects?
- Obsolete or Dead Projects
- Developer projects
- do we redirect old projects to the new location?
How do we migrate Bugs?
- What is the layout of our future bug tracker?
- Do we migrate Arch and/or AUR Packages?
- One bug tracker/project for all community/core/extra packages?
- One bug tracker/project per package?
- If we do not migrate the packages, we need to ensure consistency across bug tracker and package database
- What are the possible ways for a migration?
- Includes bugs of projects, Pacman, core/extra, community, AUR, Release Engineering, ISO
- Do a complete import of Bugzilla while migrating also all the junk we have and loosing some data such as the created date
- Do not migrate anything and keep Flyspray running in RO mode, where no new Issues are allowed and should end up in the GitLab
- Do a bug day first where we close as many bugs as possible and then do a migration afterwards within a given timeframe after which Flyspray is shut off
- Bug day (in case we do it)
- We need guidelines about how we handle (very) old bugs
- Who writes a proposal until when?
- Migration script (in case we use it)
- Do we migrate single bugs or whole lists?
- Who writes a script until when?
How do we allow access to Keycloak?
- Force 2FA? For everyone? Just staff? Just DevOps? Anyone?
- Allow brokering? (Allows for social logins via Gitlab.com, Github.com, Google to our Keycloak instance)
what goes onto Keycloak?
Do we limit DeveloperWiki:SSOMigration
Next projects to migrate to Keycloak?
The easiest projects to onboard on Keycloak would be our internal infrastructure without such as: