Difference between revisions of "Go package guidelines"

From ArchWiki
Jump to: navigation, search
(Merged introduction from User:Foxboron/Go_packaging_guidelines)
m (PKGBUILD with GOPATH and dep: Simplify by not using the var when uneeded (as in section above))
 
(19 intermediate revisions by 6 users not shown)
Line 9: Line 9:
 
== General guidelines ==
 
== General guidelines ==
  
=== Naming ===
+
=== Package naming ===
  
* For applications written in Go, use the name of the application as the package name, in lowercase.
+
For [[Go]] library modules, use {{ic|go-''modulename''}}. Also use the prefix if the package provides a program that is strongly coupled to the Go ecosystem. For other applications, use only the program name.
** Be creative if the name is already taken.
 
* For PKGBUILDS that uses the "go" tool to download the package, only add "-git" to the package name if it is not built from a tarball or a from a tagged release (but from trunk/HEAD).
 
** Similarly for mercurial packages, only add "-hg" to the package name if it is not a release-revision.
 
** Extend this pattern for other version control systems.
 
** The go tool has its own logic for which branch or tag it should use. See {{ic|go get --help}}.
 
* Consider adding the name of the author to the package name if there are several applications that are named the same, like {{AUR|dcpu16-kballard}}.
 
** In general, the most popular packages should be allowed to use the shortest or "best" name.
 
* Postfixes to the package names (like {{ic|-hg}}, {{ic|-git}} or {{ic|-svn}}) are optional if there are no official releases from the project in question. On one hand, it is common to use them when the package downloads from a VCS. On the other hand, most Go projects do not have any release-tarballs, only the repo which is used for branching/tagging the official release, if it is not ''trunk''. Also, {{ic|go get}}, which is the "official" way to install Go modules, uses the repositories directly. Use your better judgement.
 
  
=== Packaging ===
+
{{Note|The package name should be entirely lowercase.}}
  
* Go projects are either just library files, just executables or both. Choose the appropriate way of packaging them. There are several examples below.
+
=== Instructions for alternative dependency managers ===
* Some Go applications or libraries have not been updated to the latest version of Go yet.
 
** Running {{ic|go build -fix}} may often work, but it may have to be fixed by the developer. Report an issue upstream if this is the case.
 
* Several Go projects do not have a version number or a license file.
 
** Use license=('unknown') and report an issue to the developer if a license file is missing.
 
** Use version "0.1", "1" or the git-revision (or equivalent for other version control systems) if the version number is missing.
 
** Alternatively, use the current date as the version number, in this form {{ic|YYYYMMDD}}.
 
  
== Sample PKGBUILDs ==
+
This is not needed for Go 1.11 and later, unless an alternative dependency manager is required by the Go project you are packaging.
 +
 
 +
When preparing the sources before building, the following may be needed:
 +
 
 +
* Create a directory {{ic|$srcdir/gopath}} for [[Go#$GOPATH|$GOPATH]] and copy over the source to this directory.
 +
* It should be noted that this step might not be needed if the project provides a {{ic|Makefile}} for the project that sets this up.
  
=== Sample PKGBUILD for an application written in Go ===
+
{{bc|<nowiki>
 +
prepare(){
 +
  mkdir -p "gopath/src/github.com/pkgbuild-example"
 +
  ln -rTsf "${pkgname}-${pkgver}" "gopath/src/github.com/pkgbuild-example/$pkgname"
  
{{bc|<nowiki># Maintainer: NAME <EMAIL>
+
  # the dependencies can be fetched here if needed
 +
  cd "gopath/src/github.com/pkgbuild-example/$pkgname"
 +
  dep ensure
 +
}
 +
</nowiki>}}
  
pkgname=PACKAGE NAME
+
== Building ==
pkgver=1.2.3
+
 
pkgrel=1
+
There are two go packages in the repositories that you can build towards; {{Pkg|go}} and {{Pkg|go-pie}}. All packages should preferably be built towards {{Pkg|go-pie}} as this enables us to deliver secure binaries. However, as upstream might have bugs, building towards {{Pkg|go}} should be a last resort.
pkgdesc="PACKAGE DESCRIPTION"
 
arch=('x86_64')
 
url="http://SERVER/$pkgname/"
 
license=('MIT')
 
makedepends=('go')
 
options=('!strip' '!emptydirs')
 
source=("http://SERVER/$pkgname/$pkgname-$pkgver.tar.gz")
 
sha256sums=('00112233445566778899aabbccddeeff')
 
  
build() {
+
=== Flags and build options ===
  cd "$pkgname-$pkgver"
 
  
  go build
+
Most Makefiles written for go applications do not respect the {{ic|LDFLAGS}} provided by build systems, this causes Go binaries to not be compiled with RELRO. This needs to be patched into the Makefile, or the Makefile should be omitted. If there is an Makefile involved, you have 3 options to support RELRO.
}
 
  
package() {
+
* Patch Makefile
  cd "$pkgname-$pkgver"
+
* Skip Makefile completely and use {{ic|go build}}
 +
* Export {{ic|GOFLAGS}} - This is less desirable as we are dropping flags from {{ic|LDFLAGS}}.
  
  install -Dm755 "$pkgname-$pkgver" "$pkgdir/usr/bin/$pkgname"
+
For [https://reproducible-builds.org/ reproducible builds] it's important that the binaries are stripped of the build path using the {{ic|-trimpath}} flags.
  install -Dm644 LICENSE "$pkgdir/usr/share/licenses/$pkgname/LICENSE"
 
}
 
  
# vim:set ts=2 sw=2 et:</nowiki>}}
+
{{bc|<nowiki>
 +
# LDFLAGS into the GOFLAGS env variable.
 +
export GOFLAGS="-gcflags=all=-trimpath=${PWD} -asmflags=all=-trimpath=${PWD} -ldflags=-extldflags=-zrelro -ldflags=-extldflags=-znow"
  
==== Sample packages ====
+
# LDFLAGS defined in go build.
 +
go build \
 +
    -gcflags "all=-trimpath=${PWD}" \
 +
    -asmflags "all=-trimpath=${PWD}" \
 +
    -ldflags "-extldflags ${LDFLAGS}" \
 +
    .
 +
</nowiki>}}
  
* {{Pkg|gendesk}}
+
{{Note| For the sake of brevity, these flags are omitted in the examples below.}}
* {{AUR|dcpu16}}
 
  
=== Sample PKGBUILD for when only a single source file is available ===
+
=== Modern Go projects (for Go >=1.11) ===
  
{{bc|<nowiki># Maintainer: NAME <EMAIL>
+
Go 1.11 introduces [https://github.com/golang/go/wiki/Modules go modules]. This omits the need to setup {{ic|GOPATH}} and we can build directly in the directory.
  
pkgname=PACKAGE NAME
+
Projects like these have a {{ic|go.mod}} and a {{ic|go.sum}} file.  
pkgver=1.2.3
 
pkgrel=1
 
pkgdesc="PACKAGE DESCRIPTION"
 
arch=('x86_64')
 
url="http://SERVER/$pkgname/"
 
license=('GPL3')
 
makedepends=('go')
 
options=('!strip' '!emptydirs')
 
source=("http://SERVER/$pkgname/$pkgname.go")
 
sha256sums=('00112233445566778899aabbccddeeff')
 
  
build() {
+
{{Note| These builds needs {{ic|$PWD}} stripped from the binary.}}
   go build -o "$pkgname"
+
{{bc|<nowiki>
 +
build(){
 +
   cd "$pkgname-$pkgver"
 +
  go build .
 
}
 
}
 +
</nowiki>}}
  
package() {
+
=== Old Go projects (for Go <1.11) ===
  install -Dm755 "$pkgname" "$pkgdir/usr/bin/$pkgname"
 
}
 
  
# vim:set ts=2 sw=2 et:</nowiki>}}
+
When building go packages with [[Go#$GOPATH|$GOPATH]] there are a few problems one can encounter. Usually the project delivers a {{ic|Makefile}} that can be used and should be used. There are cases where you still need to setup a {{ic|$GOPATH}} in the [[PKGBUILD]]. The following snippet sets up an appropriate {{ic|GOPATH}} inside {{ic|$srcdir/gopath}} one can use to build packages. A new directory is used as some go dependency managers does wierd things if it discovers the project in the root of the {{ic|$GOPATH}}.
 +
{{Note| These builds should have {{ic|$GOPATH}} stripped from the binary.}}
 +
{{bc|<nowiki>
 +
prepare(){
 +
  mkdir -p gopath/src/github.com/pkgbuild-example
 +
  ln -rTsf $pkgname-$pkgver gopath/src/github.com/pkgbuild-example/$pkgname
 +
  export GOPATH="$srcdir"/gopath
  
==== Sample packages ====
+
  # the dependencies can be fetched here if needed
 +
  cd gopath/src/github.com/pkgbuild-example/$pkgname
 +
  dep ensure
 +
}
  
* {{AUR|gorun}}
+
build(){
 +
  export GOPATH="$srcdir"/gopath
 +
  cd gopath/src/github.com/pkgbuild-example/$pkgname
 +
  go install -v .
 +
}
 +
</nowiki>}}
  
=== Sample PKGBUILDs for Go libraries that also includes executables ===
+
Note that {{ic|install}} and {{ic|build}} and can do recursive builds if you have a {{ic|cmd/}} subdirectory with multiple binaries: {{ic|go install -v github.com/pkgbuild-example/cmd/...}}
  
==== Using ''go get'' ====
+
=== Notes about dependencies and dependency managers ===
  
This is the recommended way, instead of the method below.
+
Go packages currently use the {{ic|vendor/}} folder to handle dependencies in projects. These are usually managed by one of several dependency manager projects. If one is used for the package, this step should be performed in the [[Creating_packages#prepare.28.29|prepare()]] function of the [[PKGBUILD]].
  
Here is a way that relies on go get.
+
Dependencies are normally managed by the {{ic|go mod}} command, that comes with Go 1.11 or later.
  
You probably will not need to modify the build() or package() functions at all, only the variables at the top (pkgname etc).
+
There are also alternative dependency managers:
  
If this does not work, test with go get first.
+
* {{Pkg|dep}}
 +
* {{Pkg|godep}}
 +
* {{Pkg|glide}}
  
{{Note| Remove {{ic|/...}} if the PKGBUILD fails!}}
+
== Sample PKGBUILDs ==
  
{{bc|<nowiki># Maintainer: NAME <EMAIL>
+
=== Basic PKGBUILD ===
  
pkgname=codesearch
+
{{bc|<nowiki>
pkgver=20120515
+
pkgname=foo
 +
pkgver=0.0.1
 
pkgrel=1
 
pkgrel=1
pkgdesc="Code indexing and search written in Go"
+
pkgdesc='Go PKGBUILD Example'
 
arch=('x86_64')
 
arch=('x86_64')
url="https://github.com/google/codesearch"
+
url="https://example.org/$pkgname"
license=('BSD')
+
license=('GPL')
depends=('go')
+
makedepends=('go-pie')
makedepends=('mercurial')
+
source=("$url/$pkgname-$pkgver.tar.gz")
options=('!strip' '!emptydirs')
+
sha256sums=('1337deadbeef')
_gourl=github.com/google/codesearch
 
  
 
build() {
 
build() {
   GOPATH="$srcdir" go get -fix -v -x ${_gourl}/...
+
   cd $pkgname-$pkgver
}
+
  go build \
 
+
    -gcflags "all=-trimpath=$PWD" \
check() {
+
    -asmflags "all=-trimpath=$PWD" \
  GOPATH="$GOPATH:$srcdir" go test -v -x ${_gourl}/...
+
    -ldflags "-extldflags $LDFLAGS" \
 +
    -o $pkgname .
 
}
 
}
  
 
package() {
 
package() {
   mkdir -p "$pkgdir/usr/bin"
+
   cd $pkgname-$pkgver
   install -p -m755 "$srcdir/bin/"* "$pkgdir/usr/bin"
+
   install -Dm755 $pkgname -t "$pkgdir"/usr/bin/
 
 
  mkdir -p "$pkgdir/$GOPATH"
 
  cp -Rv --preserve=timestamps "$srcdir/"{src,pkg} "$pkgdir/$GOPATH"
 
 
 
  # Package license (if available)
 
  for f in LICENSE COPYING LICENSE.* COPYING.*; do
 
    if [ -e "$srcdir/src/$_gourl/$f" ]; then
 
      install -Dm644 "$srcdir/src/$_gourl/$f" \
 
        "$pkgdir/usr/share/licenses/$pkgname/$f"
 
    fi
 
  done
 
 
}
 
}
 +
</nowiki>}}
  
# vim:set ts=2 sw=2 et:</nowiki>}}
+
=== PKGBUILD with GOPATH and dep ===
 
 
Thanks to Rémy Oudompheng‎ for this one.
 
 
 
==== Using ''go get'' ====
 
  
Here is another way that relies on {{ic|go get}}.
+
{{bc|<nowiki>
 
+
pkgname=foo
You probably will not need to modify the build() or package() functions at all, only the variables at the top (pkgname etc).
+
pkgver=0.0.1
 
 
If this does not work, test with {{ic|go get}} first.
 
 
 
{{bc|<nowiki># Maintainer: NAME <EMAIL>
 
 
 
pkgname=PACKAGE NAME
 
pkgver=1.2.3
 
 
pkgrel=1
 
pkgrel=1
pkgdesc="PACKAGE DESCRIPTION"
+
pkgdesc='Go PKGBUILD Example'
 
arch=('x86_64')
 
arch=('x86_64')
url="http://SERVER/$pkgname/"
+
url='https://example.org/$pkgname'
license=('MIT')
+
license=('GPL')
makedepends=('go' 'git')
+
makedepends=('go-pie' 'dep')
options=('!strip' '!emptydirs')
+
source=("$url/$pkgname-$pkgver.tar.gz")
_gourl=SERVER.NET/PATH/MODULENAME
+
sha256sums=('1337deadbeef')
 +
 
 +
prepare(){
 +
  mkdir -p gopath/src/example.org/foo
 +
  ln -rTsf pkgname-$pkgver gopath/src/example.org/foo
 +
  export GOPATH="$srcdir"/gopath
 +
  cd gopath/src/example.org/foo
 +
  dep ensure
 +
}
  
 
build() {
 
build() {
   export GOROOT=/usr/lib/go
+
   export GOPATH="$srcdir"/gopath
 +
  cd gopath/src/example.org/foo
 +
  go install \
 +
    -gcflags "all=-trimpath=$GOPATH" \
 +
    -asmflags "all=-trimpath=$GOPATH" \
 +
    -ldflags "-extldflags $LDFLAGS" \
 +
    -v ./...
 +
}
  
  rm -rf build
+
check() {
  mkdir -p build/go
+
   export GOPATH="$srcdir/gopath"
  cd build/go
+
   cd gopath/src/example.org/foo
 
+
  go test ./...
  for f in "$GOROOT/"*; do
 
    ln -s "$f"
 
  done
 
 
 
  rm pkg
 
  mkdir pkg
 
  cd pkg
 
 
 
  for f in "$GOROOT/pkg/"*; do
 
    ln -s "$f"
 
  done
 
 
 
  platform=`for f in "$GOROOT/pkg/"*; do echo \`basename $f\`; done|grep linux`
 
 
 
  rm -f "$platform"
 
  mkdir "$platform"
 
  cd "$platform"
 
 
 
  for f in "$GOROOT/pkg/$platform/"*.h; do
 
    ln -s "$f"
 
  done
 
 
 
  export GOROOT="$srcdir/build/go"
 
   export GOPATH="$srcdir/build"
 
 
 
   go get -fix "$_gourl"
 
 
 
  # Prepare executable
 
  if [ -d "$srcdir/build/src" ]; then
 
    cd "$srcdir/build/src/$_gourl"
 
    go build -o "$srcdir/build/$pkgname"
 
  else
 
    echo 'Old sources for a previous version of this package are already present!'
 
    echo 'Build in a chroot or uninstall the previous version.'
 
    return 1
 
  fi
 
 
}
 
}
  
 
package() {
 
package() {
   export GOROOT="$GOPATH"
+
   install -Dm755 "gopath/bin/$pkgname" "$pkgdir/usr/bin/$pkgname"
 
+
}
  # Package go package files
+
</nowiki>}}
  for f in "$srcdir/build/go/pkg/"* "$srcdir/build/pkg/"*; do
 
    # If it's a directory
 
    if [ -d "$f" ]; then
 
      cd "$f"
 
      mkdir -p "$pkgdir/$GOROOT/pkg/`basename $f`"
 
      for z in *; do
 
        # Check if the directory name matches
 
        if [ "$z" == `echo $_gourl | cut -d/ -f1` ]; then
 
          cp -r $z "$pkgdir/$GOROOT/pkg/`basename $f`"
 
        fi
 
      done
 
      cd ..
 
    fi
 
  done
 
 
 
  # Package source files
 
  if [ -d "$srcdir/build/src" ]; then
 
    mkdir -p "$pkgdir/$GOROOT/src/pkg"
 
    cp -r "$srcdir/build/src/"* "$pkgdir/$GOROOT/src/pkg/"
 
    find "$pkgdir" -depth -type d -name .git -exec rm -r {} \;
 
  fi
 
  
  # Package license (if available)
+
== Example packages ==
  for f in LICENSE COPYING; do
 
    if [ -e "$srcdir/build/src/$_gourl/$f" ]; then
 
      install -Dm644 "$srcdir/build/src/$_gourl/$f" \
 
        "$pkgdir/usr/share/licenses/$pkgname/$f"
 
    fi
 
  done
 
 
 
  # Package executables
 
  if [ -e "$srcdir/build/$pkgname" ]; then
 
    install -Dm755 "$srcdir/build/$pkgname" \
 
      "$pkgdir/usr/bin/$pkgname"
 
  fi
 
}
 
  
# vim:set ts=2 sw=2 et:</nowiki>}}
+
* {{Pkg|dep}}
 +
* {{Pkg|gopass}}
 +
* {{Pkg|delve}}
 +
* {{Pkg|gitea}}
 +
* {{Pkg|git-lfs}}

Latest revision as of 07:54, 11 December 2018

Package creation guidelines

CLRCrossEclipseFree PascalGNOMEGoHaskellJavaKDEKernelLispMinGWNode.jsNonfreeOCamlPerlPHPPythonRRubyVCSWebWine

This document covers standards and guidelines on writing PKGBUILDs for Go.

General guidelines

Package naming

For Go library modules, use go-modulename. Also use the prefix if the package provides a program that is strongly coupled to the Go ecosystem. For other applications, use only the program name.

Note: The package name should be entirely lowercase.

Instructions for alternative dependency managers

This is not needed for Go 1.11 and later, unless an alternative dependency manager is required by the Go project you are packaging.

When preparing the sources before building, the following may be needed:

  • Create a directory $srcdir/gopath for $GOPATH and copy over the source to this directory.
  • It should be noted that this step might not be needed if the project provides a Makefile for the project that sets this up.
prepare(){
  mkdir -p "gopath/src/github.com/pkgbuild-example"
  ln -rTsf "${pkgname}-${pkgver}" "gopath/src/github.com/pkgbuild-example/$pkgname"

  # the dependencies can be fetched here if needed
  cd "gopath/src/github.com/pkgbuild-example/$pkgname"
  dep ensure
}

Building

There are two go packages in the repositories that you can build towards; go and go-pie. All packages should preferably be built towards go-pie as this enables us to deliver secure binaries. However, as upstream might have bugs, building towards go should be a last resort.

Flags and build options

Most Makefiles written for go applications do not respect the LDFLAGS provided by build systems, this causes Go binaries to not be compiled with RELRO. This needs to be patched into the Makefile, or the Makefile should be omitted. If there is an Makefile involved, you have 3 options to support RELRO.

  • Patch Makefile
  • Skip Makefile completely and use go build
  • Export GOFLAGS - This is less desirable as we are dropping flags from LDFLAGS.

For reproducible builds it's important that the binaries are stripped of the build path using the -trimpath flags.

# LDFLAGS into the GOFLAGS env variable.
export GOFLAGS="-gcflags=all=-trimpath=${PWD} -asmflags=all=-trimpath=${PWD} -ldflags=-extldflags=-zrelro -ldflags=-extldflags=-znow"

# LDFLAGS defined in go build.
go build \
    -gcflags "all=-trimpath=${PWD}" \
    -asmflags "all=-trimpath=${PWD}" \
    -ldflags "-extldflags ${LDFLAGS}" \
    .
Note: For the sake of brevity, these flags are omitted in the examples below.

Modern Go projects (for Go >=1.11)

Go 1.11 introduces go modules. This omits the need to setup GOPATH and we can build directly in the directory.

Projects like these have a go.mod and a go.sum file.

Note: These builds needs $PWD stripped from the binary.
build(){
  cd "$pkgname-$pkgver"
  go build .
}

Old Go projects (for Go <1.11)

When building go packages with $GOPATH there are a few problems one can encounter. Usually the project delivers a Makefile that can be used and should be used. There are cases where you still need to setup a $GOPATH in the PKGBUILD. The following snippet sets up an appropriate GOPATH inside $srcdir/gopath one can use to build packages. A new directory is used as some go dependency managers does wierd things if it discovers the project in the root of the $GOPATH.

Note: These builds should have $GOPATH stripped from the binary.
prepare(){
  mkdir -p gopath/src/github.com/pkgbuild-example
  ln -rTsf $pkgname-$pkgver gopath/src/github.com/pkgbuild-example/$pkgname
  export GOPATH="$srcdir"/gopath

  # the dependencies can be fetched here if needed
  cd gopath/src/github.com/pkgbuild-example/$pkgname
  dep ensure
}

build(){
  export GOPATH="$srcdir"/gopath
  cd gopath/src/github.com/pkgbuild-example/$pkgname
  go install -v .
}

Note that install and build and can do recursive builds if you have a cmd/ subdirectory with multiple binaries: go install -v github.com/pkgbuild-example/cmd/...

Notes about dependencies and dependency managers

Go packages currently use the vendor/ folder to handle dependencies in projects. These are usually managed by one of several dependency manager projects. If one is used for the package, this step should be performed in the prepare() function of the PKGBUILD.

Dependencies are normally managed by the go mod command, that comes with Go 1.11 or later.

There are also alternative dependency managers:

Sample PKGBUILDs

Basic PKGBUILD

pkgname=foo
pkgver=0.0.1
pkgrel=1
pkgdesc='Go PKGBUILD Example'
arch=('x86_64')
url="https://example.org/$pkgname"
license=('GPL')
makedepends=('go-pie')
source=("$url/$pkgname-$pkgver.tar.gz")
sha256sums=('1337deadbeef')

build() {
  cd $pkgname-$pkgver
  go build \
    -gcflags "all=-trimpath=$PWD" \
    -asmflags "all=-trimpath=$PWD" \
    -ldflags "-extldflags $LDFLAGS" \
    -o $pkgname .
}

package() {
  cd $pkgname-$pkgver
  install -Dm755 $pkgname -t "$pkgdir"/usr/bin/
}

PKGBUILD with GOPATH and dep

pkgname=foo
pkgver=0.0.1
pkgrel=1
pkgdesc='Go PKGBUILD Example'
arch=('x86_64')
url='https://example.org/$pkgname'
license=('GPL')
makedepends=('go-pie' 'dep')
source=("$url/$pkgname-$pkgver.tar.gz")
sha256sums=('1337deadbeef')

prepare(){
  mkdir -p gopath/src/example.org/foo
  ln -rTsf pkgname-$pkgver gopath/src/example.org/foo
  export GOPATH="$srcdir"/gopath
  cd gopath/src/example.org/foo
  dep ensure
}

build() {
  export GOPATH="$srcdir"/gopath
  cd gopath/src/example.org/foo
  go install \
    -gcflags "all=-trimpath=$GOPATH" \
    -asmflags "all=-trimpath=$GOPATH" \
    -ldflags "-extldflags $LDFLAGS" \
    -v ./...
}

check() {
  export GOPATH="$srcdir/gopath"
  cd gopath/src/example.org/foo
  go test ./...
}

package() {
  install -Dm755 "gopath/bin/$pkgname" "$pkgdir/usr/bin/$pkgname"
}

Example packages