The goal is to move away from using svn internally to manage PKGBUILDs to using git. This enables developers to use tools they are more familiar with and facilitate future outside contributions.
- Devtools repo: https://github.com/Foxboron/devtools-1/tree/morten/git
- Dbscripts repo: https://github.com/Foxboron/dbscripts/tree/morten/git-support
- Devops Issues: https://gitlab.archlinux.org/archlinux/infrastructure/-/issues/135
- Name: User:foxboron
- Name: and whoever more are still invested in this.
It's important to mention gitlab as we are going for a "federated" approach. The implication here is that we still need one way to define what the actual PKGBUILD source is. For the sake of simplicity in this proposal, we envision a centralized package group on the upcoming gitlab instance. The structure would follow from the current repository layout.
https://gitlab.archlinux.org/groups/packages/ - core - linux - pacman - ... - extra - community - multilib
The access control for this repository should be the same as it currently is. Developers has access to core and extra. Trusted Users have access to community. And multilib is granted on a need-to-need basis.
This allows Developers and Trusted Users to create new canonical package repositories in the package group without devops support, and allows for smooth operation.
The default access for these repositories should be,
- No issue creation
- No Merge Requests.
- Maintainers should be able to enable or disable this on a case-by-case basis.
We still have our bugtracker for handling issues, features and bugs. And not everyone want the added commitment of handling pull-requests directly on the repo. Thus it should be decided maintainer by maintainer themselves.
This also allows us to *know* where repositories are for the sake of script integrations later on.
The git interaction of this layout should be fairly straight forward. Remotes are cheap, and so are new repositories. The canonical repositories in the package group is where dbscripts intersects with the maintainer, along with a rewritten asp tool. Besides this, anything should be supported. If the maintainer wishes to have a subtree, submodule or some other mono repository structure it should be simple to construct this from the single lonely repositories for easier management.
When releasing packages, *signed* tags of the pkgver needs to be done. This allows dbscripts to attest the validity of the package release, along with the signed package tarball the maintainer has created. Equally, devtools should aid the packager in these situations.
State repository layout
dbscripts would maintain a separate git repository responsible for keeping track of the package version between releases. Each pkgbase is represented as a file denoting the package version.
$ tree /srv/dbscripts dbscripts ├── core-x86_64 │ ├── bash │ ├── linux │ └── vim ├── extra-x86_64 ├── multilib ├── ...
bash 4.4.023-1 f19949abae19c11687679663c488a3321e138e4b
This repository is tagged with an incremented number, or a date, and represent the state of the Arch Linux repositories over time. It would keep track of the repository by implementing the same commits as we have in todays svn repository.
The following Git log is from the experimental repository.
commit b27b2aff2022712031ee818b69f4eef1697e7d7b (HEAD -> master, origin/master) Author: Morten Linderud <firstname.lastname@example.org> Date: Wed May 27 00:23:00 2020 +0200 db-remove: hello removed by fox: commit 2c7c56e3a1757eb47a9cbac71f0ba9b0bb552b49 Author: Morten Linderud <email@example.com> Date: Wed May 27 00:22:59 2020 +0200 db-move: moved hello from [core] to [extra] (x86_64): commit b942f04f78d3260972f6307023243f57978e7892 Author: Morten Linderud <firstname.lastname@example.org> Date: Tue May 26 22:54:05 2020 +0200 updated hello-1-1 in [core-x86_64]:
dbscripts has been refactored so the SVN interaction. The largest different is the removal of
packages seperation. It's easier to provide one set of tools and have
dbscripts attempt to guess where the upstream repository is.
/community/db-update has been replaces by
/packages/db-update and so on.
Addition/Update of a package
- Ensure tag is signed by key from
- Ensure pkgver is always larger with
- Never allow someone to replace the git commit with the same tag
- Increment the tag of the
Removal of a package
db-remove would ensure packages are removed, and the entry in the state repository is removed.
Moving of a package
db-move would locate the package, move between the given repository and record the movement in the state repository.
The current changes can be found in Foxboron/devtools.
- Ensure signde tags are created with
- Ensure a checksummed PKGBUILD matches the one in the built package.
.SRCINFOis created and commited.
- Enforce commits from the master branch.
For users to fetch PKGBUILDs from our package repository we have a few options.
- We can drop frontend tooling and point people towards the gitlab interface
aspto work with the new structure and simplify the script
- Provide a new script
Currently there is nothing decided which option we are going with.
Testing server: repos.linderud.dev
Deployment repository: Foxboron/dbscripts
Currently all accounts are setup as orion.
For every directory in our svn repositority (svn-community/svn-packages) a git repository has to be created. Anthraxx has created a svn to git migration script. Manjaro also mirrors our svn2git repository on Gitlab already, and we consider using their script for this migration. 
- How do we populate the dbscripts database?
- How do we tag previous releases?
- Code search - Do we want to construct a monorepo to facilitate repo-wide grepping?
Software requiring migration
- arch-commits - svn hook and perl script maintained in infrastructure.git
- archweb - populate_signoffs hook, fetches commit message from svn to be shown in signoffs). Also https://archlinux.org/svn/ should be removed.
- infrastructure - disable svnlog, arch-svntogit
Gemini git cache
Currently we are cloning all required git repositories upon dbscripts usage. Because this is a network operation there are concerns this would be slow when dealing with larger package migrations. The ideal solution would be to have a mirror of all git repositories locally on gemini and ensure these are updated when new commits are pushed to gitlab.
This is the list of intended security goals of this migration:
- Enforce git signed commits
- Verify git signed commits in a git hook and against the keys in the pacman keyring.
- Disallow force pushes
- Disallow deploy keys - relevant as we need to ensure malicious commits are not present.
- Use git tags to match the checksum of the uploaded PKGBUILD with the built package