User talk:Foxboron/Go packaging guidelines

As mentioned on Reddit earlier, I had some feedback about the Go packaging guidelines.

The Go toolchain does not support symlinks inside of GOPATH. Doing so causes unwanted behavior in some operations, notably the skipping of tests.
Tests should likely be run with /... at the end in order to run an entire project's tests and not just what's in the base repo. This thankfully won't rerun tests from vendor/ as of recent Go versions.
Moving vendor/ up into $GOPATH/src seems a bit awkward, and I think was done due to the symlink problem. I personally think modifying the given structure should be avoided.

Just to give an example of what I'm suggesting, here's a modified version of dep's PKGBUILD.

_url='github.com/golang/dep'

prepare() {
  export GOPATH="${srcdir}"
  mkdir -p "$GOPATH/src/${_url}"
  cp -r "${srcdir}/${pkgname}-${pkgver}"/* "$GOPATH/src/${_url}"
}

build() {
  go install -v "${_url}/cmd/dep"
}

check() {
  go test "${_url}/..."
}

package() {
  install -Dm755 "$GOPATH/bin/dep" "${pkgdir}/usr/bin/dep"
  install -Dm644 "$GOPATH/src/${_url}/LICENSE" "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE"
}

Instead of linking, I copied the extracted files into the right place, and in the build/check functions, I invoke install on the only binary being packaged (otherwise I'd suggest "${_url}/...") and test on all packages in the repo. Unfortunately, dep does a lot of integration-style tests, which makes the overall testing time pretty high (versus some 4 seconds of tests as the current PKGBUILD runs now). I'm not sure what the policy is on check functions.

I'm not a Bash scripting or PKGBUILD expert, and I'm not suggesting swapping my version in for the current one as presented above, but I wanted to point out a few Go-related tidbits that might be helpful if you didn't already know them.

Jakebailey (talk) 02:38, 26 May 2018 (UTC)Reply[reply]

So, i made some progress (finally).

I think this is a prefect balance. We keep the symlink for incremental builds and we don't move things around.

prepare(){
  mkdir -p src/github.com/golang
  ln -rTsf "${pkgname}-${pkgver}" src/github.com/golang/dep
}

build(){
  export GOPATH="${srcdir}"
  cd src/github.com/golang/dep
  go install -gcflags "all=-trimpath=${GOPATH}/src" -asmflags "all=-trimpath=${GOPATH}/src" ./cmd/...
}

check(){
  cd src/github.com/golang/dep
  go test ./...
}

package(){
  install -Dm755 bin/dep "${pkgdir}/usr/bin/dep"

  cd "${pkgname}-${pkgver}"
  install -Dm644 LICENSE "${pkgdir}/usr/share/licenses/${pkgname}/LICENSE"
}

Foxboron (talk) 17:14, 29 July 2018 (UTC)Reply[reply]

Yo! Sorry for not answering quickly :)

The symlink problem was new to me. I did that for a few reason: 1) Avoid cp at all cost. 2) I thought go install would fetch missing dependencies from the internet. Which should be avoided at all cost during build. Looking at go install now I'm unsure if my assumption was wrong. So I plan to rewrite some packages and see.

As for check, thats a mistake. You are completely correct. The dep pkgbuild is wrong and I'll change it.

Foxboron (talk) 12:07, 1 June 2018 (UTC)Reply[reply]

You might be able to get away with a symlink, so long as you don't somehow invoke the Go path walker. I think you were hitting this before when it was looking for dependencies and getting down to the vendor dir, but aren't when doing other operations. This may or may not be a good option, but I also played with adding the source tarball to noextract, then extracting directly to the fake GOPATH in prepare. With vgo on the horizon, out-of-GOPATH builds should be supported and all of this won't be needed. (Though I think that vgo's choice in dependency version selection will clash extremely poorly with any plans to package individual libraries.)

go install doesn't access the internet to grab dependencies, no, though I think it may if you explicitly specify the -i flag. go install (as of Go 1.10) is essentially just go build into $GOPATH/bin, but with the ability to traverse downward into other directories (i.e. you can do go install github.com/golang/dep/... and get a few more binaries than just dep). Previously, go install was the only way to get the Go compiler to cache intermediate steps, but the most recent version allows go build to as well. I'd personally use go install, mainly because of that ability to traverse the repo and build binaries into a common place, and that it's also what go get does without the -d flag after fetching source (see here).

Jakebailey (talk) 14:35, 1 June 2018 (UTC)Reply[reply]

After doing some testing I agree. go install $_url/cmd/... is neat. And solves a few issues with my PKGBUILDs actually. I think I'll fix the current guidelines, removing the symlink hack and stick with cp.

I appreciate the feedback and feel free to make any edits on the page where you see fit :)

Foxboron (talk) 11:54, 11 June 2018 (UTC)Reply[reply]

I think I'd just do NoExtract+bsdtar, or mv. Just because symlinks are issueful (really, golang????) doesn't mean you should need to sprinkle extra files all over.

This does raise questions about the ability to use incremental builds at all, using mv requires cleaning up existing files first, and bsdtar requires --strip-components -C ... to remove the leading ${pkgname}-${pkgver}/ in most cases (and creating the directory ahead of time). -- Eschwartz (talk) 12:32, 11 June 2018 (UTC)Reply[reply]