Difference between revisions of "VCS package guidelines"

From ArchWiki
Jump to: navigation, search
m (Tips: - Ellipsis: A three-dot symbol used to show an incomplete statement. :))
(Prototypes: the abs package was removed from repos)
 
(204 intermediate revisions by 44 users not shown)
Line 1: Line 1:
 
[[Category:Package development]]
 
[[Category:Package development]]
[[it:VCS PKGBUILD Guidelines]]
+
[[it:VCS package guidelines]]
[[zh-CN:VCS PKGBUILD Guidelines]]
+
[[ja:VCS PKGBUILD ガイドライン]]
[[zh-TW:VCS PKGBUILD Guidelines]]
+
[[zh-hans:VCS package guidelines]]
{{Package Guidelines}}
+
[[zh-hant:VCS package guidelines]]
 +
{{Package guidelines}}
  
[[Wikipedia:Revision_control|Version control systems]] can be used for retrieval of source code for both usual statically versioned packages and latest (trunk) version of packages. This article covers both cases.
+
[[Wikipedia:Revision_control|Version control systems]] can be used for retrieval of source code for both usual statically versioned packages and latest (trunk) version of a development branch. This article covers both cases.
  
 
== Prototypes ==
 
== Prototypes ==
The [[ABS]] package provides prototypes for [[cvs]], [[svn]], [[git]], [[mercurial]], and [[Wikipedia:darcs|darcs]] [[PKGBUILD]]s. When {{Pkg|abs}} is installed, you can find them in {{ic|/usr/share/pacman}}. Latest versions can be found in the [https://projects.archlinux.org/abs.git/tree/prototypes prototypes directory in the ABS Git repository].
+
 
 +
Use only the PKGBUILD prototypes provided in the {{Pkg|pacman}} package ({{ic|PKGBUILD-split.proto}}, {{ic|PKGBUILD-vcs.proto}} and {{ic|PKGBUILD.proto}} in {{ic|/usr/share/pacman}}).
  
 
== Guidelines ==
 
== Guidelines ==
* Properly suffix {{Ic|pkgname}} with {{Ic|-cvs}}, {{Ic|-svn}}, {{Ic|-hg}}, {{Ic|-darcs}}, {{Ic|-bzr}}, {{Ic|-git}} etc. If the package tracks a moving development trunk it should be given a suffix. If the package fetches a release from a VCS tag then it should not be given a suffix. Use this rule of thumb: if the output of the package depends on the time at which it was compiled, append a suffix; otherwise do not.
 
  
* A VCS package may be updated as and when needed to adopt changes to the build system, including ammendments to dependencies, URL, sources, etc. If the revision number remains the same after such an update, but produces a resulting binary which is different, increasing the {{Ic|pkgrel}} is mandatory. If both the revision number and the resulting binary remain the same, {{Ic|pkgrel}} should be kept intact. There is no need to update the VCS package just to accommodate a revision bump, but one may choose to do so.
+
* Suffix {{Ic|pkgname}} with {{Ic|-cvs}}, {{Ic|-svn}}, {{Ic|-hg}}, {{Ic|-darcs}}, {{Ic|-bzr}}, {{Ic|-git}} etc. unless the package fetches a specific release.
 +
 
 +
* If the resulting package is different after changing the dependencies, URL, sources, etc. increasing the {{Ic|pkgrel}} is mandatory. Touching the {{ic|pkgver}} is not.
 +
 
 +
* {{Ic|--holdver}} can be used to prevent [[makepkg]] from updating the {{ic|pkgver}} (see: [https://www.archlinux.org/pacman/makepkg.8.html makepkg(8)])
  
* When [[makepkg]] is run, by default it will check for newer revisions and then update the {{Ic|pkgver}} in the PKGBUILD.  Look at {{Ic|--holdver}} in [http://www.archlinux.org/pacman/makepkg.8.html man makepkg] if you want otherwise.  {{Ic|--holdver}} only works for cvs and svn, which allow checkout of older revisions.
+
* Include what the package conflicts with and provides (e.g. for {{AUR|fluxbox-git}}: {{Ic|1=conflicts=('fluxbox')}} and {{Ic|1=provides=('fluxbox')}}).
  
* Check for package conflicts.  For example ''fluxbox-svn'' will conflict with ''fluxbox''.  In this case, you need to use {{Ic|1=conflicts=('fluxbox')}}.
+
* {{Ic|1=replaces=()}} generally causes unnecessary problems and should be avoided.
  
* Use the {{Ic|provides}} field so that packages that require the non-VCS package can be installed ({{Ic|1=provides=('fluxbox')}}).
+
* When using the cvsroot, use {{Ic|anonymous:@}} rather than {{Ic|anonymous@}} to avoid having to enter a blank password or {{Ic|anonymous:password@}}, if one is required.
  
* You should AVOID using {{Ic|1=replaces=...}} as it generally causes unnecessary problems.
+
* Include the appropriate VCS tool in {{Ic|1=makedepends=()}} ({{pkg|cvs}}, {{pkg|subversion}}, {{pkg|git}}, ...).
  
* When using/defining the cvsroot, use {{Ic|anonymous:@}} rather than {{Ic|anonymous@}} to avoid a password prompt and having to enter a blank password ''OR'' use {{Ic|anonymous:password@}} if a password is required.
+
* Because the sources are not static, skip the checksum in {{ic|1=md5sums=()}} by adding {{ic|'SKIP'}}.
  
* Don't forget to include the appropriate VCS tool (cvs, subversion, git, ...) in {{Ic|1=makedepends=...}}.
+
=== VCS sources ===
 +
{{Note|Pacman 4.1 supports the following VCS sources: {{ic|bzr}}, {{ic|git}}, {{ic|hg}} and {{ic|svn}}. See the {{ic|fragment}} section of {{ic|man PKGBUILD}} or [https://www.archlinux.org/pacman/PKGBUILD.5.html PKGBUILD(5)] for a list of supported VCS.}}
  
* To preserve the integrity of the checked-out code consider copying the original build directory if you have to make edits. For example, having checked out source code to {{ic|src/$_cvsmod}} from {{ic|$startdir}} you can use:
+
Starting with {{Pkg|pacman}} 4.1, the VCS sources should be specified in the {{ic|1=source=()}} array and will be treated like any other source. {{ic|makepkg}} will clone/checkout/branch the repository into {{ic|$SRCDEST}} (same as {{ic|$startdir}} if not set in [https://www.archlinux.org/pacman/makepkg.conf.5.html makepkg.conf(5)]) and copy it to {{ic|$srcdir}} (in a specific way to each VCS). The local repository is left untouched, thus invalidating the need for a {{ic|-build}} directory.
  
mkdir src/$_cvsmod-build
+
The general format of a VCS {{ic|1=source=()}} array is:
+
  source=('[folder::][vcs+]url[#fragment]')
cd src/$_cvsmod-build
 
  ../$_cvsmod/configure
 
  
or:
+
* {{ic|folder}} (optional) is used to change the default repository name to something more relevant (e.g. than {{ic|trunk}}) or to preserve the previous sources.
 +
* {{ic|vcs+}} is needed for URLs that do not reflect the VCS type, e.g. {{ic|<nowiki>git+http://some_repo</nowiki>}}.
 +
* {{ic|url}} is the URL to the distant or local repository.
 +
* {{ic|#fragment}} (optional) is needed to pull a specific branch or commit. See {{ic|man PKGBUILD}} for more information on the fragments available for each VCS.
  
  cp -r src/$_cvsmod src/$_cvsmod-build
+
An example Git source array:
cd src/$_cvsmod-build
+
  <nowiki>source=('project_name::git+http://project_url#branch=project_branch')</nowiki>
  
* With the introduction of the [[AUR]], it is most important to avoid using backtick execution to create package variables. makepkg will automatically bump the {{Ic|pkgver}} anyway when building the package (unless {{Ic|--holdver}} is used).
+
=== The pkgver() function ===
  
== Tips ==
+
The {{ic|pkgver}} autobump is now achieved via a dedicated {{ic|pkgver()}} function. This allows for better control over the {{ic|pkgver}}, and maintainers should favor a {{ic|pkgver}} that makes sense. To use {{ic|pkgver()}}, you still need to declare the {{ic|pkgver}} variable with the most recent value. makepkg will invoke function {{ic|pkgver()}}, and update variable {{ic|pkgver}} accordingly.
* You should make sure that there are no VCS directories and files left over in your package. If there are, you may want to remove them, by adding a command similar to this one at the end of the the package() script:
+
 
 +
It is recommended to have following version format: ''RELEASE.rREVISION'' where ''REVISION'' is a monotonically increasing number that uniquely identifies the source tree (VCS revisions do this). If there are no public releases and no repository tags then zero could be used as a release number or you can drop ''RELEASE'' completely and use version number that looks like ''rREVISION''. If there are public releases but repo has no tags then the developer should get the release version somehow e.g. by parsing the project files.
 +
 
 +
The revision number delimiter ("r" right before REVISION) is important. This delimiter allows to avoid problems in case if upstream decides to make its first release or uses versions with different number of components. E.g. if at revision "455" upstream decides to release version 0.1 then the revision delimiter preserves version monotonicity - {{ic|0.1.r456 > r454}}. Without the delimiter monotonicity fails - {{ic|0.1.456 < 454}}.
 +
 
 +
See [https://projects.archlinux.org/pacman.git/tree/proto/PKGBUILD-vcs.proto#n33 PKGBUILD-vcs.proto] for generic examples showing the intended output.
 +
 
 +
==== Git ====
 +
 
 +
Using the most recent annotated tag reachable from the last commit:
 +
 
 +
{{hc|<nowiki>
 +
pkgver() {
 +
  cd "$pkgname"
 +
  git describe --long | sed 's/\([^-]*-g\)/r\1/;s/-/./g'
 +
}</nowiki>|
 +
2.0.r6.ga17a017
 +
}}
 +
 
 +
Using the most recent un-annotated tag reachable from the last commit:
 +
 
 +
{{hc|<nowiki>
 +
pkgver() {
 +
  cd "$pkgname"
 +
  git describe --long --tags | sed 's/\([^-]*-g\)/r\1/;s/-/./g'
 +
}</nowiki>|
 +
0.71.r115.gd95ee07
 +
}}
 +
 
 +
In case if the git tag does not contain dashes then one can use simpler sed expression {{ic|sed 's/-/.r/;s/-/./'}}.
 +
 
 +
If tag contains a prefix, like {{ic|v}} or project name then it should be cut off:
 +
 
 +
{{hc|<nowiki>
 +
pkgver() {
 +
  cd "$pkgname"
 +
  # cutting off 'foo-' prefix that presents in the git tag
 +
  git describe --long | sed 's/^foo-//;s/\([^-]*-g\)/r\1/;s/-/./g'
 +
}</nowiki>|
 +
6.1.r3.gd77e105
 +
}}
 +
 
 +
If there are no tags then use number of revisions since beginning of the history:
 +
 
 +
{{hc|<nowiki>
 +
pkgver() {
 +
  cd "$pkgname"
 +
  printf "r%s.%s" "$(git rev-list --count HEAD)" "$(git rev-parse --short HEAD)"
 +
}</nowiki>|
 +
r1142.a17a017
 +
}}
 +
 
 +
Version and only commit/revision number (SHA1 omitted; however, without a SHA1 quick referencing of an exact revision is lost if not mindful of versioning):
 +
 
 +
git describe --long | sed -r 's/-([0-9,a-g,A-G]{7}.*)//' | sed 's/-/./'
 +
 
 +
Both methods can also be combined, to support repositories that start without a tag but get tagged later on (uses a bashism):
 +
 
 +
{{hc|<nowiki>
 +
pkgver() {
 +
  cd "$pkgname"
 +
  ( set -o pipefail
 +
    git describe --long 2>/dev/null | sed 's/\([^-]*-g\)/r\1/;s/-/./g' ||
 +
    printf "r%s.%s" "$(git rev-list --count HEAD)" "$(git rev-parse --short HEAD)"
 +
  )
 +
}</nowiki>|
 +
0.9.9.r27.g2b039da  # if tags exist
 +
r1581.2b039da      # else fallback
 +
}}
 +
 
 +
==== Subversion ====
 +
 
 +
{{hc|<nowiki>
 +
pkgver() {
 +
  cd "$pkgname"
 +
  local ver="$(svnversion)"
 +
  printf "r%s" "${ver//[[:alpha:]]}"
 +
}</nowiki>|
 +
r8546
 +
}}
 +
 
 +
{{Note|If the project has releases you should use them instead of the {{ic|0.}}.}}
  
rm -rf $(find "$pkgdir" -type d -name ".svn")
+
==== Mercurial ====
  
* When using Git, one can speed up the cloning operation using the {{Ic|1=--depth=1}} parameter. This creates a shallow clone, and has only the last change history - since histories are unimportant for builds most of the time.
+
{{hc|<nowiki>
 +
pkgver() {
 +
  cd "$pkgname"
 +
  printf "r%s.%s" "$(hg identify -n)" "$(hg identify -i)"
 +
}</nowiki>|
 +
r2813.75881cc5391e
 +
}}
  
git clone git://hostname.dom/project.git --depth=1
+
==== Bazaar ====
  
* It's possible to create the package also from a branch other than the master. To do so add {{Ic|1=--branch branch_name}} after the first {{Ic|1=git clone}}, in this way:
+
{{hc|<nowiki>
 +
pkgver() {
 +
  cd "$pkgname"
 +
  printf "r%s" "$(bzr revno)"
 +
}</nowiki>|
 +
r830
 +
}}
  
git clone "$_gitroot" "$_gitname" --branch branch_name
+
==== Fallback ====
Remember to save package with a different name, for example {{Ic|1= pkgname-branchname-git}}, in order to avoid confusion with the package from the branch master.
 
  
* Copy paste script when building from repo
+
In case no satisfactory {{ic|pkgver}} can be extracted from the repository, the current date can be used:
If you are lazy here is a sample script when making git-based PKGBUILDs.
 
  
  cd "$srcdir"
+
{{hc|<nowiki>
  msg "Connecting to GIT server..."
+
pkgver() {
   if [ -d $_gitname ] ; then
+
   date +%Y%m%d
    cd $_gitname && git pull origin
+
}</nowiki>|
    msg "The local files are updated."
+
20130408
  else
+
}}
    git clone --depth=1 $_gitroot $_gitname
 
  fi
 
  msg "GIT checkout done or server timeout"
 
  
* Temporary build directories: When using Git, and where you need to create a separate build directory (e.g., for building/compiling), you should avoid copying over the {{Ic|1=.git}} directory located in the parent folder because it contains history information that Git uses internally. With repos with thousands of commits, this .git directory will contain upwards of hundreds of MiB of useless commit history that has *nothing* to do with the current working tree. The only time you'd need to copy over the .git directory itself is when you need to build from a specific, older commit (which is generally never the case as the point of a VCS PKGBUILD is to pull from the latest bleeding edge commit). Thus, instead of
+
Although it does not identify source tree state uniquely, so avoid it if possible.
  
rm -rf "$srcdir/$_gitname-build"
+
== Tips ==
cp -R "$srcdir/$_gitname" "$srcdir/$_gitname-build" # copy everything, including the useless .git folder
 
cd "$srcdir/$_gitname-build"
 
make # build/compile from source
 
  
you should do
+
=== Git Submodules ===
 +
Git submodules are a little tricky to do. The idea is to add the URLs of the submodules themselves directly to the sources array and then reference them during prepare(). This could look like this:
  
rm -rf "$srcdir/$_gitname-build"
+
{{bc|<nowiki>
cd "$srcdir/$_gitname" && ls -A | grep -v .git | xargs -d '\n' cp -r -t ../$_gitname-build # do not copy over the .git folder
+
source=("git://somewhere.org/something/something.git"
cd "$srcdir/$_gitname-build"
+
        "git://somewhere.org/mysubmodule/mysubmodule.git")
make # build/compile from source
 
  
to cut down on build time and disk usage.
+
prepare() {
 +
  cd something
 +
  git submodule init
 +
  git config submodule.mysubmodule.url $srcdir/mysubmodule
 +
  git submodule update
 +
}
 +
</nowiki>}}

Latest revision as of 16:08, 17 May 2017

Package creation guidelines

CLRCrossEclipseFree PascalGNOMEGoHaskellJavaKDEKernelLispMinGWNode.jsNonfreeOCamlPerlPHPPythonRubyVCSWebWine

Version control systems can be used for retrieval of source code for both usual statically versioned packages and latest (trunk) version of a development branch. This article covers both cases.

Prototypes

Use only the PKGBUILD prototypes provided in the pacman package (PKGBUILD-split.proto, PKGBUILD-vcs.proto and PKGBUILD.proto in /usr/share/pacman).

Guidelines

  • Suffix pkgname with -cvs, -svn, -hg, -darcs, -bzr, -git etc. unless the package fetches a specific release.
  • If the resulting package is different after changing the dependencies, URL, sources, etc. increasing the pkgrel is mandatory. Touching the pkgver is not.
  • Include what the package conflicts with and provides (e.g. for fluxbox-gitAUR: conflicts=('fluxbox') and provides=('fluxbox')).
  • replaces=() generally causes unnecessary problems and should be avoided.
  • When using the cvsroot, use anonymous:@ rather than anonymous@ to avoid having to enter a blank password or anonymous:password@, if one is required.
  • Include the appropriate VCS tool in makedepends=() (cvs, subversion, git, ...).
  • Because the sources are not static, skip the checksum in md5sums=() by adding 'SKIP'.

VCS sources

Note: Pacman 4.1 supports the following VCS sources: bzr, git, hg and svn. See the fragment section of man PKGBUILD or PKGBUILD(5) for a list of supported VCS.

Starting with pacman 4.1, the VCS sources should be specified in the source=() array and will be treated like any other source. makepkg will clone/checkout/branch the repository into $SRCDEST (same as $startdir if not set in makepkg.conf(5)) and copy it to $srcdir (in a specific way to each VCS). The local repository is left untouched, thus invalidating the need for a -build directory.

The general format of a VCS source=() array is:

source=('[folder::][vcs+]url[#fragment]')
  • folder (optional) is used to change the default repository name to something more relevant (e.g. than trunk) or to preserve the previous sources.
  • vcs+ is needed for URLs that do not reflect the VCS type, e.g. git+http://some_repo.
  • url is the URL to the distant or local repository.
  • #fragment (optional) is needed to pull a specific branch or commit. See man PKGBUILD for more information on the fragments available for each VCS.

An example Git source array:

source=('project_name::git+http://project_url#branch=project_branch')

The pkgver() function

The pkgver autobump is now achieved via a dedicated pkgver() function. This allows for better control over the pkgver, and maintainers should favor a pkgver that makes sense. To use pkgver(), you still need to declare the pkgver variable with the most recent value. makepkg will invoke function pkgver(), and update variable pkgver accordingly.

It is recommended to have following version format: RELEASE.rREVISION where REVISION is a monotonically increasing number that uniquely identifies the source tree (VCS revisions do this). If there are no public releases and no repository tags then zero could be used as a release number or you can drop RELEASE completely and use version number that looks like rREVISION. If there are public releases but repo has no tags then the developer should get the release version somehow e.g. by parsing the project files.

The revision number delimiter ("r" right before REVISION) is important. This delimiter allows to avoid problems in case if upstream decides to make its first release or uses versions with different number of components. E.g. if at revision "455" upstream decides to release version 0.1 then the revision delimiter preserves version monotonicity - 0.1.r456 > r454. Without the delimiter monotonicity fails - 0.1.456 < 454.

See PKGBUILD-vcs.proto for generic examples showing the intended output.

Git

Using the most recent annotated tag reachable from the last commit:

pkgver() {
  cd "$pkgname"
  git describe --long | sed 's/\([^-]*-g\)/r\1/;s/-/./g'
}
2.0.r6.ga17a017

Using the most recent un-annotated tag reachable from the last commit:

pkgver() {
  cd "$pkgname"
  git describe --long --tags | sed 's/\([^-]*-g\)/r\1/;s/-/./g'
}
0.71.r115.gd95ee07

In case if the git tag does not contain dashes then one can use simpler sed expression sed 's/-/.r/;s/-/./'.

If tag contains a prefix, like v or project name then it should be cut off:

pkgver() {
  cd "$pkgname"
  # cutting off 'foo-' prefix that presents in the git tag
  git describe --long | sed 's/^foo-//;s/\([^-]*-g\)/r\1/;s/-/./g'
}
6.1.r3.gd77e105

If there are no tags then use number of revisions since beginning of the history:

pkgver() {
  cd "$pkgname"
  printf "r%s.%s" "$(git rev-list --count HEAD)" "$(git rev-parse --short HEAD)"
}
r1142.a17a017

Version and only commit/revision number (SHA1 omitted; however, without a SHA1 quick referencing of an exact revision is lost if not mindful of versioning):

git describe --long | sed -r 's/-([0-9,a-g,A-G]{7}.*)//' | sed 's/-/./'

Both methods can also be combined, to support repositories that start without a tag but get tagged later on (uses a bashism):

pkgver() {
  cd "$pkgname"
  ( set -o pipefail
    git describe --long 2>/dev/null | sed 's/\([^-]*-g\)/r\1/;s/-/./g' ||
    printf "r%s.%s" "$(git rev-list --count HEAD)" "$(git rev-parse --short HEAD)"
  )
}
0.9.9.r27.g2b039da  # if tags exist
r1581.2b039da       # else fallback

Subversion

pkgver() {
  cd "$pkgname"
  local ver="$(svnversion)"
  printf "r%s" "${ver//[[:alpha:]]}"
}
r8546
Note: If the project has releases you should use them instead of the 0..

Mercurial

pkgver() {
  cd "$pkgname"
  printf "r%s.%s" "$(hg identify -n)" "$(hg identify -i)"
}
r2813.75881cc5391e

Bazaar

pkgver() {
  cd "$pkgname"
  printf "r%s" "$(bzr revno)"
}
r830

Fallback

In case no satisfactory pkgver can be extracted from the repository, the current date can be used:

pkgver() {
  date +%Y%m%d
}
20130408

Although it does not identify source tree state uniquely, so avoid it if possible.

Tips

Git Submodules

Git submodules are a little tricky to do. The idea is to add the URLs of the submodules themselves directly to the sources array and then reference them during prepare(). This could look like this:

source=("git://somewhere.org/something/something.git"
        "git://somewhere.org/mysubmodule/mysubmodule.git")

prepare() {
  cd something
  git submodule init
  git config submodule.mysubmodule.url $srcdir/mysubmodule
  git submodule update
}