Status

anthraxx wrote a svn to git migration script

https://github.com/anthraxx/arch-svn-package-to-git

Migration Plan

Clean up remaining TODOs where reasonable
- 2 TODOs in commitpkg
- some in tests/lib/common.sh about svn related code
Check if all common usage cases have tests
Check everything is locked properly
- repos that are touched should be locked
- the internal git db should be locked

Set up a test environment (on nymeria?) (this needs more thought)
- Duplicate svn or test with a smaller number of packages?
- Verify that normal/old usage to release/create and update a package really works
- Test -debug package and release to multiple repos

Migrate production
- Package commitpkg and archrelease
  - Package them in devtools or in a dedicate package that is required by devtools?
  - Determine failure mode if old devtools are used
- Create $pkgbase/tags/$tag for each package in SVN
- Create matching internal dbscripts db (How?)
- ABS?

Migrate to git
- Implement TODOs in db-functions related to git
- Migrate repos (this needs more thought)

Goal

Decouple dbscripts from svn to make it faster and simpler. Moving lots of packages is awefully slow right now.

svn tracks the PKGBUILDS and tags refering to certain PKGBUILDs. dbscripts maintains an internal, separate database which tracks in which repo a given package is (identified by a pkgname and a tag).

Definitions

tag: Identifier for a particular pkgbuild + support files; git tag/svn repos subdir; "$epoch:$pkgver-$pkgrel"
package: A single .pkg.tar.xz file. -debug and split packages count as single packages.

dbscripts DB

This is not important to packagers, it's internal to dbscripts and only tracks which packages are in a given repo for history purposes and might be used to update abs.

The db looks like the current abs tree in /var/abs except that every package in there is a git submodule.

The db also has to handle multiple arches (community-i686, multilib-x86_64).

Note: git submodules is awefully slooow even with just under 5k submodules. Need to evaluate git subtree.

Example

/var/abs/core/pacman is a submodule pointing to the pacman PKGBUILD git repo and the tag that has been released to core.

Assuming pacman-shared is a split package originating from the pacman package, /var/abs/core/pacman-shared will be a symlink to /var/abs/core/pacman. Possible performance problems. If not possible replace submodule with copy of the repo content? (This is also possible if we use svn instead of git for the repos). Just an idea, not yet asked on ML.

I believe dbscripts wouldn't need a db to perform the functions outlined below so we can talk about this later.

Executables

Note: The "all" arch is an alias for all possible arches (currently i686, x86_64), but this only really affects -any packages. commitpkg -a all pkg1-i686.pkg.tar.xz pkg2-any.pkg.tar.xz will not release pkg1 to x86_64, but it will release pkg2 to i686 and x86_64. Similar behaviour can be expected for other commands that accept an arch.

commitpkg [-a $arch] (from devtools)

create a tag and upload packages to the server
-any packages default to be released to all arches
if target arch is supplied we only release to one arch (also counts for -any packages)
support for directly supplying the commit message (commitpkg "update to version 1.2.3")
-debug packages should go into a special debug repo, not the normal repo

commitpkg [-a $arch] pkg1.tar.xz pkg2.tar.xz

if called with arguments it will only release those, nothing else
-any packages will be release to all arches unless -a is used

db-add $repo $arch pkg1.pkg.tar pkg2.pkg.tar

check if the tag exists in the package repo (extract pkgbase from package)
check if the user has permission to release to the repo
calls repo-add on the package

db-update

calls db-add for every package in the user's staging directory
possibly do sanity checks

db-move $srcrepo $dstrepo $arch pkg1 pkg2

move packages between repos

db-remove $repo $arch pkg1 pkg2

calls repo-remove

Example

pacman, core repo
pacman-debug (automatic debug split), debug repo
pacman-shared (split any arch package), core repo
pacman-contrib (normal split), extra repo

All these are create by building a single PKGBUILD.

db-* scripts still run on the server, but for simplicity I'll ignore that below.

Releasing

Note: Just create a ./release script in your package repo to do that if a simple foopkg doesn't cut it.

corepkg pacman-xxx.pkg.tar.xz pacman-debug-xxx.tar.xz pacman-shared-xxx.tar.xzA
extrapkg pacman-contrib-xxx.tar.xz
db-update

Removing

db-remove core all pacman pacman-shared pacman-contrib pacman-debug
- removes the packages from both arches

moving from testing to core:

db-move testing core i686 pacman pacman-shared pacman-contrib pacman-debug
- moves only i686, leaves x86_64
- to move both arches at once just use all instead of i686