Talk:Go package guidelines

From ArchWiki
Latest comment: 8 April 2023 by Franklin Yu in topic about -linkmode=external

Go Modules and Caching

I was recently building a package and noticed that it errored in CI but worked locally. It turned out I had a cached version of a dependency that had been removed upstream. This made me wonder if it wouldn't be good practice to recommend setting: GOCACHE="${srcdir}/cache" or similar before downloading packages so that doing a clean build also destroys the cache.

SamWhited (talk) 05:44, 22 January 2019 (UTC)Reply[reply]

I'm not sure if this is desirable for most users. When I'm not building in a chroot, I want makepkg to use my local cache to build with. This saves me time when building.

The problem you experienced is a side-effect of the golang community relying on third party hosting for dependencies that aren't immutable, like github. Projects like Athens address this by providing a proxy that caches any dependency ever requested of it. If run locally using such a proxy also saves a lot of time downloading dependencies if running in a clean chroot. It can also be set up to run for your CI so that a changed repo doesn't stop your build anymore. This requires exporting GOPROXY in your build to point at the proxy endpoint.

Alaskanarcher (talk) 04:39, 9 May 2019 (UTC)Reply[reply]

Go Modules, -git packages, and pkgver

For anyone maintaining -git packages of Go tools, the following can be put in pkgver() to output the version that Go Modules uses if you don't have any semver compatible tags. This way, your packages can have a version that someone requiring your tool can drop in their go.mod file:

   git show --abbrev-commit --abbrev=12 --date='format:%G%m%d%H%M%S' --pretty=format:v0.0.0_%cd_%h --no-patch HEAD

SamWhited (talk) 05:56, 22 January 2019 (UTC)Reply[reply]

EDIT: and if you want to automatically pick up the tag if they start using them, something like this will work:

   TAG=$(git describe --tags --abbrev=0 2>/dev/null || echo "v0.0.0" | sed s/-/_/g)
   SUFFIX=$(git show --abbrev-commit --abbrev=12 --date='format:%G%m%d%H%M%S' --pretty=format:%cd_%h --no-patch HEAD)
   printf '%s_%s' "$TAG" "$SUFFIX"

SamWhited (talk) 15:55, 22 January 2019 (UTC)Reply[reply]

More detail about best practices for LDFLAGS and patching makefiles

The page currently suggests patching the Makefile or "Export GOFLAGS - This is less desirable as we are dropping flags from LDFLAGS."

Could someone explain how LDFLAGS should be used when running go build? That would help me understand why the use of GOFLAGS is less desirable, and how I might patch an existing Makefile to make use of LDFLAGS.

Alaskanarcher (talk) 04:20, 9 May 2019 (UTC)Reply[reply]

Additionally, can anyone comment on whether CGO_LDFLAGS should be set for projects that use CGO?

Alaskanarcher (talk) 04:40, 9 May 2019 (UTC)Reply[reply]


CGO_LDFLAGS is more or less mentioned in the guidelines now Foxboron (talk) 11:48, 15 March 2020 (UTC)Reply[reply]

Add section suggesting the use of GOPROXY when building packages in a clean chroot

When building in a clean chroot, `go build` must fetch all dependencies again. Using a go modules proxy locally like Athens dramatically speeds up subsequent builds by allowing all dependencies to be cached for future builds in the clean chroot. I find this extremely helpful when working on my golang PKGBUILDs.

Alaskanarcher (talk) 19:40, 9 May 2019 (UTC)Reply[reply]

GOPROXY shouldn't be used in proper PKGBUILDs. And there is no way for you to whitelist this ENV variable when building with devtools. Setting this inside the PKGBUILD would break for anyone not running a proxy. This is not going to be part of the package guidelines. Foxboron (talk) 11:47, 15 March 2020 (UTC)Reply[reply]

Guidelines should mention proper usage of go mod

The guidelines speak about `go mod` but then make no mention on how to use that properly. Is it `go mod vendor` as I see in some of the official packages or `go mod download`? Svenstaro (talk) 17:49, 14 January 2020 (UTC)Reply[reply]

Mention usage of -mod=vendor

The official(?) go modules wiki mentions `-mod=vendor` but I don't see a consistent usage on that on our packages. It should be mentioned on when to use that. Svenstaro (talk) 17:56, 14 January 2020 (UTC)Reply[reply]

From experience, -mod=vendor should be used when package sources (tarballs, VCS repositories, etc) include a vendor directory containing modules.txt. Despite upstream documentation stating that the -mod flag can be safely omitted if the go version in the go.mod file is higher than 1.14, it would be simpler to differentiate between the flag's options (readonly vs. vendor).
Grawlinson (talk) 00:05, 4 October 2021 (UTC)Reply[reply]

about -linkmode=external

It's better to let go decide the linkmode. Go programs that don't use net os/user or runtime/cgo let a warning when this flag is used.

https://golang.org/src/cmd/cgo/doc.go#L1000
By default, cmd/link will decide the linking mode as follows: if the only
packages using cgo are those on a list of known standard library
packages (net, os/user, runtime/cgo), cmd/link will use internal linking
mode. Otherwise, there are non-standard cgo packages involved, and cmd/link
will use external linking mode. The first rule means that a build of
the godoc binary, which uses net but no other cgo, can run without
needing gcc available. The second rule means that a build of a
cgo-wrapped library like sqlite3 can generate a standalone executable
instead of needing to refer to a dynamic library. The specific choice
can be overridden using a command line flag: cmd/link -linkmode=internal or
cmd/link -linkmode=external.

Try compiling this simple go program that doesn't use cgo.

main.go
package main

func main() {
	println("main")
}
$ go mod init main
$ go build -ldflags=-linkmode=external
# main
loadinternal: cannot find runtime/cgo
$ go build

—This unsigned comment is by ENV25 (talk) 06:14, 11 January 2022‎ (UTC). Please sign your posts with ~~~~!Reply[reply]

The reason why they are provided is because you need the external linker to provide RELRO and binary hardening, these are not supported by the internal linker. Now, we could argue if they are important or not, but generally we want the hardening we can get.
Foxboron (talk) 08:39, 11 January 2022 (UTC)Reply[reply]
What if cgo is disabled? In that case, does RELRO do anything? --Franklin Yu (talk) 02:03, 8 April 2023 (UTC)Reply[reply]

Recommend GOENV=off

Go stores environment variable overrides in ~/.config/go/env. This means that, if not building in a clean chroot, options specified using environment variable might not take effect.

I think we should mention that in Go package guidelines#Flags and build options and recommend adding export GOENV=off.

https://github.com/golang/go/issues/30411

https://github.com/golang/go/issues/46840

ENV25 (talk) 08:39, 23 September 2022 (UTC)Reply[reply]