Talk:VCS package guidelines

pkgver function for hg based on tags

I recent came across a way with hg to show the most recent tag, as well as the number of commits from this tag (similar to the output of git describe.)

pkgver() {
  cd local_repo
  hg log -r . --template '{latesttag}.{latesttagdistance}.{node|short}\n'
}

3.0.1.40.ee9a2543fcd6

Please could this be included in the page.

Garyvdm (talk) 09:03, 23 July 2013 (UTC)

shorten hg version

To prevent long package file name, It is proper to use this format

pkgver() {
  cd $_repo
  _id=$(hg identify -i)
  echo $(hg identify -n).${_id:0:4}
}

--Dlin (talk) 05:30, 26 August 2013 (UTC)

fossil

pkgver() {
  cd local_repo
  _id=$(cat manifest.uuid 2>/dev/null)
  echo ${_id:0:4}
}

--Dlin (talk) 05:36, 26 August 2013 (UTC)

set -o pipefail

This change mentioned set -o ... as one of the reasons for reverting, nevertheless one example using set -o pipefail still remains: VCS_package_guidelines#Git (the last example in that section). -- Lahwaacz (talk) 17:41, 6 March 2015 (UTC)

Guidelines on updating version numbers

There are a few version control AUR packages out there which use the correct combined format of release and git commit. However, sometimes the maintainers refuse to update the default pkgver upon new releases of the software, stating that it's unnecessary since it's updated automatically during installation anyway. While this is true, I personally feel it should still be the maintainer's responsibility to update the default pkgver upon actual releases (not individual commits) as otherwise a user will never receive an automatic update and instead has to force-reinstall all(!) such packages regularly.

Should we add this to the guidelines or how do others feel about this?

—This unsigned comment is by Airblader (talk) 09:51, 10 September 2015‎. Please sign your posts with ~~~~!

There's no official update mechanism for AUR packages, so this argument only applies to those using unofficial AUR helpers. Otherwise there's soname bumps; if the package is a library, then pkgver should likely be updated. If it is a library the package depends on, I don't know of a reliable way as pkgrel is stuck to 1 on VCS packages. After all, a soname bump could happen between releases or even commits. -- Alad (talk) 11:00, 10 September 2015 (UTC)

If you're using a VCS package, you should be paying attention to what is happening upstream and choose when you rebuild the package. Managing them the same way as release packages makes no sense.

Scimmia (talk) 13:25, 10 September 2015 (UTC)

Can you explain why it doesn't make sense to manage them the same way? One example is the package i3-git. As of writing this, it's version in the AUR is 4.10.2-xxxxx. The current i3 release is 4.10.4. So why *shouldn't* the package maintainer feel responsible to update it to 4.10.4-xxxxx? It's part of the Arch "feel" to be close to current development of software and I feel that if you maintain a package, you are responsible to update its version even if the actual installation process doesn't change. Sure I can rebuild my git packages, but that's really time intensive. I also cannot monitor all of those packages upstream with reasonable effort. This is exactly what a package management system has version numbers for and keeping them up to date is exactly what package maintainers are for, IMHO. Airblader (talk) 11:39, 11 September 2015 (UTC)

VCS packages will always build the latest revision, regardless of what pkgver on the AUR contains. If a package maintainer bumps pkgver of a VCS package, there is no assurance that the users will actually build the same (working) revision, hence the bumping makes no sense. If you need to track multiple git repos, use something like repocheck to find out if you actually need to rebuild the package. I'm sure there are similar tools for other VCS, or just write your own. -- Lahwaacz (talk) 12:13, 11 September 2015 (UTC)

I don't think that bumping makes no sense just because you build the latest version and not the one stated by pkgver. This is *always* the case, even if the version isn't bumped. The bump isn't meant to build exactly that version, it is meant to trigger an automatic update in the user's package management system that doesn't require checking out the repository and verifying it themselves. Again, IMHO this is what version numbers are for. If we don't care about such packages triggering an update or having any kind of comparable version number, what's the point in even giving them a (meaningful) number? We might as well just set pkgver=1 and be done with it, no? So why all the fancy pkgver() functions for VCS packages? I don't see how the repocheck you linked to is relevant for this. I don't have the repositories of my VCS packages laying around (only in /tmp right after I update them and only until I reboot). I'd like to approach this with a pragmatic argument: I hope we can agree that triggering the update for official releases of the package is some benefit. Given that, what's the *harm* in recommending to update pkgver? If there's an opinion that it makes no sense, that's one thing, but as long as it doesn't harm I think the small benefit sounds like reason enough to me. It's not like updating the pkgver of a package is difficult work. Airblader (talk) 13:20, 11 September 2015 (UTC)

"I hope we can agree that triggering the update for official releases of the package is some benefit." We can't. I really see no benefit there at all. "Given that, what's the *harm* in recommending to update pkgver?" The harm comes from annoying people using those AUR helpers you mentioned. They have correctly chosen when they want to rebuild that package, now their helper is complaining all of the time even though they don't want to rebuild right now. Trigger automatic rebuilds of VCS packages is rarely a good thing, as there are no upsides and real downsides.

Your argument that it is really time intensive to rebuild your VCS packages makes no sense here, as you have to rebuild them either way. If you're unable to monitor upstream, why are you using a VCS package anyway? If you have no idea what's going on, you have no idea what you're getting when you build/rebuild it.

Scimmia (talk) 13:38, 11 September 2015 (UTC)

I don't think that you choose VCS packages because you don't want your package management system to annoy you with updates. If that's something you don't want, then you don't use one at all (why would I choose to not receive updates for *that one* package, but for all 500 others?). IMHO, you choose VCS packages because you want to be close to current development, either "just because" or because you need(ed) it for something else. Also, sometimes there just is *only* the VCS version and no package for the release. Also, if you don't want to receive updates about a certain package, you can always ignore it in the config. I don't think it's good to deny *all* users the notification of updates just because *some* might not want them.

I also can't agree with "triggering automatic rebuilds of VCS packages is rarely a good thing". I never see upront what I actually build, so there's always that "risk". You don't install VCS packages if you want the release version. Furthermore, in practice, it seems rare that rebuilding a VCS package causes any issues since they usually point to the master branches which in most projects have some QA before merging.

Your point about having to rebuild them anyway and hence not saving time is not correct. You're arguing that I should just rebuild all VCS packages all the time just in case there was an update. I'm suggesting the information whether a package needs to be rebuilt is transported in the pkgver. This saves a *drastic* amount of time as I'm not speculating. Of course I can check the Github page of each package's upstream URL before rebuilding it, but this puts the work of versioning the package onto each and every user rather than the package maintainer. Again, if this is the desired behavior, we should recommend that VCS packages just set pkgver=1. I don't see why they need to provide any version info if it's completely unnecessary that it ever be updated. Airblader (talk) 13:48, 11 September 2015 (UTC)

"I'm suggesting the information whether a package needs to be rebuilt is transported in the pkgver." This is really the heart of the matter. This is completely and totally wrong for VCS packages. Whether the package needs a rebuild is up to your judgement, not the pkgver.

There's nothing wrong with setting pkgver=1 (unless the actual version is less than 1). Some AUR package already do this, using "LATEST". The important thing is that the correct pkgver appears on the build package.

Scimmia (talk) 13:59, 11 September 2015 (UTC)

I just don't agree that it's "completely and totally" wrong. Yeah, sure, I can always decide to rebuild it myself. No one is taking this option away from anybody. But setting the pkgver when the actual release changes means the user is notified of this without having to care about it themselves (all n of them). Anyway, I don't think this is going anywhere. To me, letting the pkgver rot destroys a major point to why I use a package manager in the first place and makes me wonder why that person wants to be a package maintainer if they don't maintain the package, but clearly there's very different views on this. Thanks for the discussion anyway, at least I know now that there's no common view on it so there's no point in pushing for a change of guideline. Airblader (talk) 22:58, 11 September 2015 (UTC)

Sorry for reopening, I didn't make it in time to reply earlier...

I'll start by saying that I agree that VCS package maintainers shouldn't bump pkgver manually. However, suggesting to use a less confusing pkgvar value would seem sensible to me, e.g. we could indeed give "LATEST" as an example.

— Kynikos (talk) 04:04, 13 September 2015 (UTC)

I also wanted to try to give @Airblader another perspective. You have mentioned 2 different cases:

• A package has both a "stable" and a "vcs" version, e.g. i3 and i3-git^AUR: why are you using i3-git^AUR in this case if you want to follow the normal upstream releases?
• A package only has a "vcs" version, e.g. wmfs2-git^AUR (dead, I know, but take it as an example): in this case the AUR maintainer can't be blamed for not bumping pkgver manually, so the problem is actually that there's nobody maintaining a "stable" PKGBUILD, and you could be the one filling that gap!

— Kynikos (talk) 04:04, 13 September 2015 (UTC)

Sorry for the necroposting, but what about AUR packages that have explicitly as a dependency a VCS package (even if the stable version exists), or more commonly packages for which upstream doesn't bother to make discrete releases?

ThePirate42 (talk) 18:20, 21 January 2022 (UTC)

What about them? Scimmia (talk) 18:26, 21 January 2022 (UTC)

Well, since sometime there isn't any choice other than using a VCS package, I feel like there should be some way (not necessarily the pkgver) for PKGBUILD mantainers to, at least, signal that some important change has happened upstream; and this in a way that is inspectable by automated means. Of course this should be optional for mantainers, but it seems to me that it could save a lot of time for people that need a VCS package, but would otherwise be perfectly happy to just interact with non VCS AUR pages that can be easily parsed by scripts and not with various, uneven upstream pages.

ThePirate42 (talk) 23:14, 21 January 2022 (UTC)

Git submodules: sometimes not all submodules are needed

As per this writeup by Earnestly (which I understand is the initial source of that section): https://ptpb.pw/r/N6-z.md Some packages shouldn't try to checkout all submodules, because it is wasteful and unneeded. It might be nice to expand on that section by mentioning this. -- Eschwartz (talk) 14:19, 12 July 2017 (UTC)

Add notice to this page?

The Arch User Repository page indicates:

Note: VCS packages are not considered out of date when the pkgver changes, so please do not flag them as the maintainer will merely unflag the package and ignore you. AUR maintainers should not commit mere pkgver bumps.

Perhaps this should be included on this page as well, to inform new maintainers that they should not be making pkgver bumps for these packages. That being said, there should probably be a way to update the pkgver shown by the AUR web UI, or disable it entirely for VCS packages, in order to prevent others from believing a package is out of date and flagging it. JacobHenner (talk) 11:33, 17 November 2017 (UTC)

GitHub API usage

Some git repos are really huge (ex. QGIS) and git-clone process takes a lot of time and space. So, here is an example how to use GH API to obtain pkgver:

_orgname=foo
_pkgname=bar
_branch=master
_gh_api_url="https://api.github.com/repos/${_orgname}/${_pkgname}"
...
makedepends=('jq')
...
source=("https://github.com/${_orgname}/${_pkgname}/archive/${_branch}.tar.gz")
sha256sums=('SKIP')

pkgver() {
  read -r tag_name tag_sha <<<$(curl -s "${_gh_api_url}/tags" | \
    jq -r '.[0]|[.name,.commit.sha]|@sh' | sed "s/'//g")
  head=$(curl -s "${_gh_api_url}/git/refs/heads/${_branch}" | \
    jq -r '.object.sha')
  count=$(curl -s "${_gh_api_url}/compare/${tag_sha}...${head}" | \
    jq -r '.total_commits')

  printf "%s.r%s.g%.8s" "${tag_name}" "${count}" "${head}"
}

0.1.r7.g47de1a51

—This unsigned comment is by Sikmir (talk) 2019-06-28. Please sign your posts with ~~~~!

makepkg still requires cloning the whole source code repository to build the package. In fact, the source code fetching happens before the pkgver function. IMO, this option does not seem to have any use for packaging in Arch. By the way, if you use the binary jq, make sure to add jq to makedepends array. -- Josephgbr (talk) 11:37, 28 June 2019 (UTC)

No! It doesn't require cloning, see source variable, there is only gzipped snapshot. Sikmir (talk) 16:18, 28 June 2019 (UTC)

I didn't see the tarball in source before, but in my opinion a PKGBUILD with a branch/tag's tarball as source does not classify as a VCS package and, therefore, should not be included to this page. I understand GitHub's snapshot tarball represents the last commit of the software, but still you are not using a VCS. Personally, since Arch Linux has these well defined type of packages, normal/non-VCS package with defined version and VCS packages with dynamic version, I would make my dynamic versioning packages as VCS package due to the advantage of just updating the source code repository instead of re-downloading it all. -- Josephgbr (talk) 18:08, 28 June 2019 (UTC)

Add GPG Signature verification note

It is possible to verify the git sources using gpg. This is documented in "man PKGBUILD", but most people do not know this. It would be easy to have this information in the tip section of this guideline.

source=(<url>?signed)

validpgpkeys=('<keyid>')

NicoHood (talk) 07:54, 3 June 2020 (UTC)

Manually constructed 'git describe' output should be forward-compatible

The recommendation for when no tags exist isn't forward-compatible wrt adding tags -- note the absence of a 'g' prefix in the SHA.

Gesh (talk) 15:22, 20 December 2020 (UTC)

It could be added for consistency, but it really doesn't matter. The versions are incremented by the revision, and if a tag gets added to the front of that, that's obviously an upgrade. Functionally, nothing would change.

Scimmia (talk) 15:26, 20 December 2020 (UTC)

This inconsistency bugs me too. However, I'd rather remove the g from tagged versions instead of adding it to untagged versions. That would be consistent with the suggestions for mercurial. Also, commands like git log 1596c1f do not accept the g prefix. So removing the prefix seems only logical.

In all sed commands, change s/\([^-]*-g\)/r\1/ to s/\([^-]*-\)g/r\1/.

Also; I'd suggest changing the combined function as follows. This does not require changing the settings to set -o pipefail and keeps both versioning alternatives consistent with a single change of the sed command.

pkgver() {
  cd "srandom"
  {
    git describe --long --tags 2>/dev/null ||
    echo "$(git rev-list --count HEAD)-g$(git describe --always)"
  } | sed 's/\([^-]*-\)g/r\1/;s/-/./g'
}

Schaetzc (talk) 14:23, 10 September 2021 (UTC)

Guidelines: (mostly) cosmetic improvements

Continuing this discussion, after the structural changes were finished, I made a separate edit for the cosmetic and technical changes that had been proposed alongside them, which has been reverted.

Let's discuss the problems with the content as it stands and how to make it better.

As you can see from the diff linked above, the changes break down to roughly six sections.

The first is a simple substitution of pkgname with pkgbase.

Problem: pkgname doesn't cover PKGBUILDs for split packages.

Solution: pkgbase covers both single and split pacakges.

The second is a simplification of what is to be done about pkgrel and pkgver.

Problem: The list of specific things that warrant a change to pkgver or pkgrel is confusing, inherently incomplete, and long.

Solution: Maintainers should be directed to prioritize updating pkgrel whenever the PKGBUILD itself has changed. If we are to tell them to update pkgver at all, we need to be clear and concise about when that is. Last I checked, consenus was that it is never really necessary to update pkgver for a VCS package.

The third regards conflicts and provides.

Problem: The pronoun "what" is unclear, while the example indicates that the objective of this section is to inform maintainers that the repository version of the package is to be included to these arrays.

Solution: Explicitly state that the repository version of the package needs to be appended to these arrays.

The fourth regards replaces.

Problem: This is a cryptic warning that doesn't communicate anything about what replaces is for or what kind of problems using it could cause, which will almost certainly result in people getting confused and trying it out anyway

Solution: Explain what replaces actually does and when it might be useful. An alternative approach might be to move the clarification to PKGBUILD#replaces (using the word "deprecate" somewhere) and link there with a shorter, lest cryptic statment or remove this section altogether (we don't really need to say "see this? don't touch it." when we could just ignore it to begin with).

The fifth is a totally cosmetic edit, changing makedepends=() to makedepends.

Problem: foobar=(), as a convention is ugly, unnecessary in this context, and inconsistent with other pages in the wiki.

Solution: foobar is sufficent and easier to process.

The sixth regards skipping checksums for VCS sources.

Problem: md5sums=() does not account for the various kinds of checksums that may be used and simply "adding" 'SKIP' may place it in the wrong order if a VCS package has multiple sources.

Solution: use a wildcard to indicate that there are various kinds of checksums, and be more technical about what to do with 'SKIP'.

Please see the diff linked above if you'd like to review my changes that were reverted, or see them on the full page draft made for the previous discussion. quequotion (talk) 15:23, 11 January 2023 (UTC)

For the first, pkgname *does* cover both split and non-split packages. If it's specified separately, it doesn't really matter, the pkgname is what's important here. This change is completely wrong.

For the second, no, updating the pkgrel when there's a newer pkgver makes no sense at all. It won't be picked up by anything checking for updates, making it pretty useless. The consensus is to not *just* bump the pkgver, bumping it when changing things in the PKGBUILD is normal.

For the third, this could work, but it still needs to be clear that all conflicts and provides need to be included, not just the non-vcs version.

For the fourth, this could be done much more simply by just linking to the PKGBUILD page, but leaving what we already have here. The point is to stop people from saying it replaces the non-vcs version, which people did a lot before this warning was put here.

The fifth and sixth are something for the wiki maintainers who put together the style guidelines to address.

Scimmia (talk) 15:40, 11 January 2023 (UTC)

>If it's specified separately

This invites someone to forget to append their -vcs to one of the separate package routines; appending it to pkgbase ensures it gets appended to all of them.

>updating the pkgrel when there's a newer pkgver makes no sense

In the context of VCS packages, how often is updating pkgver really relevant? There's a tip immediately following this suggesting maintainers use --holdver. On top of that, can we actually account for how "anything checking for updates" works on the wiki? That seems dangerously close to legitimizing AUR helpers. This has been a contentious issue on this page for a long time; it would be helpful to have a clear policy statement here.

For the third, how about:

• If available, append the repository version of the package to conflicts and provides (e.g. for fluxbox-git^AUR: conflicts=('fluxbox') and provides=('fluxbox')).

For the fourth, how about:

• Do not append the repository version of the package to replaces, which is only for deprecation.

quequotion (talk) 17:31, 11 January 2023 (UTC)

1. The package function names are based on the pkgname, not the pkgbase, so your reasoning here doesn't make any sense to me.

2. Bumping the pkgrel all of the time with an old pkgver is useless in all cases. Period.

3. It's not just about repository packages, and it's not always just <pkgname> vs <pkgname>-git. There can be various different builds, which is why the wording that's there is generic but adds the example to a common case.

4. Also not just about repo packages, although that's where it causes most issues.

Scimmia (talk) 16:51, 13 January 2023 (UTC)

1. I was thinking pkgname derives from pkgbase when defined; apparently the other way around. OK.

2. That's totally opposed to what has been policy as laid out on this page and argued on this talk page since long before I ever used Archlinux. The point with VCS packages is that pkgver does not matter *in the AUR* most of the time: users of the package should be keeping track of upstream changes on their own, while it's maintainer should only update when changes to the PKGBUILD are required, ie changes that denote a new pkgver. VCS package maintainers are generally supposed to ignore pkgver unless they have a good reason to update it.

3 & 4 True, but the wider scope of these arrays should be handled on the PKGBUILD page. These blurbs, being on this page, are specifically for the context of VCS packages. ie, their intent is to instruct maintainers what additional considerations there are for VCS packages, which happens to be the interaction with repository versions of the packages.

quequotion (talk) 00:45, 16 January 2023 (UTC)

About git submodules in combination with `$SRCDEST`

As it stands there are a lot of projects that use submodules to pull in sources, and sometimes it just not possible to depend on system libraries, either because the build configuration upstream is using is not capable, it requires extensive patching etc. In any case, you can find yourself in two particular situations.

1. The first one is using a fork of the original project as a submodule. This can cause issues with the shared SRCDEST option for makepkg the remotes will be different and pulling it in source array will complain about it.
2. The second one is the program being built renames the submodule directory into something different than the repository name or uses a sub-directory. In this case renaming the repository in the source array can cause multiples of the same git repository in $SRCDEST.

Ideally I would like to see two additions to the guidelines.

1. One is that if the submodule pulls in from a fork, the source array should clone into a $pkgname specific directory. I think a format like <repo_name>-$pkgname::git+https://<source_url>.git would suffice. Example: xz-pcsx2::git+https://github.com/PCSX2/xz.git
2. The other one would be to add something along the lines of the following in tips and tricks

   _submodules=(
       xz-pcsx2::3rdparty/xz/xz
       googletest::3rdparty/gtest
       Vulkan-Headers::3rdparty/vulkan-headers
       rcheevos::3rdparty/rcheevos/rcheevos
   )
   for submodule in ${_pcsx2_submodules[@]}; do
       git submodule init "${submodule#*::}"
       git submodule set-url "${submodule#*::}" "$srcdir"/"${submodule%::*}"
       git -c protocol.file.allow=always submodule update "${submodule#*::}"
   done

The snippet above follows the same pattern as the `source` array or splitting the name of the source and the path of the submodule with a double colon `::`. The first part is the name as specified in the source array, the second part is the path of the submodule in relative to the directory of program being built.. Example source array:

source=(
   git+https://github.com/PCSX2/pcsx2.git
   xz-pcsx2::git+https://github.com/PCSX2/xz.git
   git+https://github.com/google/googletest.git
   git+https://github.com/KhronosGroup/Vulkan-Headers.git
   git+https://github.com/RetroAchievements/rcheevos.git
)

--LoathingKernel (talk) 14:15, 25 March 2023 (UTC)

Honestly, I think a single git submodule update --init --recursive is much more convinient than doing all that stuff manually. I don't know why it is not even mentioned in the article.

Using it, you only need 1 source - main repository. Submodules are resolved and pulled automatically. It works like a charm, at least with GitHub.

And it perfectly solves the described problem, because in this case submodules are stored in-place inside the main repo.

Hanabishi (talk) 15:25, 25 March 2023 (UTC)

Using git submodule update --init --recursive breaks both the convention of $SRCDEST for shared sources handling and makepkg --allsource.

It also creates unnecessary network traffic, server load and time waste because the whole git tree has to be fetched for repeated builds. Using --depth 1 can help with these but it still breaks makepkg functionality.

I am aware of other build system, such as rust's cargo or go's also break this convention, but that is probably a much larger discussion than this one.

In my opinion there shouldn't be any network access in prepare()

LoathingKernel (talk) 15:50, 25 March 2023 (UTC)

Using --filter=tree:0 mitigates the traffic issue almost completely. (Btw it is also very useful for VCS packages in general, too bad makepkg's GITFLAGS varable is still not in the stable release.)

Regaring the convention, IMO complying with it simply does not worth such amount of headache that the manual method brings. And considering the problem you described, it turns into a major PITA.

Hanabishi (talk) 16:08, 25 March 2023 (UTC)

I see what you mean with --filter-tree:0, that's a good alternative.

For the sake of completeness I want to mention another problematic case, which should also be mentioned. makepkg clones from the local repo making remote a reference to the locally cloned bare repository. This will make submodules specified using relative paths fail. Maybe this workaround should also be mentioned somewhere.

git remote set-url origin https://github.com/RPCS3/rpcs3.git

git -c protocol.file.allow=always submodule update --init --depth=1

LoathingKernel (talk) 18:16, 25 March 2023 (UTC)

Yeah, that's kind of a drawback. But every particular package can be tuned individually.

Also there may be other good ways. I personally not very deep into git documentation.

Hanabishi (talk) 18:49, 25 March 2023 (UTC)

I almost wonder if alot of this could be avoided by giving makepkg / the build environments a way to parse .gitmodules if it exists in a project

The advantage being that it would auto handle any odd naming conventions and see recursive modules.

Weirdbeard (talk) 17:24 25 March 2023 (UTC)

Naming convention

How is the suffix determined for other version control systems? e.g. fossil, pijul, dat ... Do we just use the full name of the VCS?

Eclairevoyant (talk) 11:21, 27 June 2023 (UTC)

Git pkgver format consistency

Current guideline for pkgver with git:

• with annotated tags: git describe --long --tags --abbrev=7 | sed 's/\([^-]*-g\)/r\1/;s/-/./g'
• with un-annotated tags: git describe --long --abbrev=7 | sed 's/\([^-]*-g\)/r\1/;s/-/./g'
• without tags: printf "r%s.%s" "$(git rev-list --count HEAD)" "$(git rev-parse --short=7 HEAD)"

This seems to translate in fairly standard template, which I would formalize with something similar to: [major[[.minor].patch]].r{rev_count_since_tag_or_root}.g{hash_last_commit} but first two methods keep the "g" prefix before commit hash while last method do not add this "g". I would argue that prefixing "g" or not in all the cases would make pkgver more consistent across different packages. Tmms (talk) 14:29, 6 August 2024 (UTC)