Talk:VCS package guidelines

Numeric vs Date Standard in Git pkgver

I'd like to see a standard emerge in AUR Git packages, but standards have to work down to the lowest common denominator. Only a small minority of GitHub projects properly tag their releases, so there can be only two LCD standards for Git packages: the Numeric standard, e.g. ("r1581.2b039da") and the Date standard, e.g. ("2014-01-01"). The date standard is much more readable, and actionable, and has the additional benefit of being backwards compatible with older Git packages.

I don't know if any standard can emerge even if we wanted it to, but please allow me to at least add the Date standard to the VCS package guidelines wiki (as opposed to scrubbing it, :).

Using the commit date would be incorrect because it doesn't refer to a specific commit and the dates are not always sequential. It would be even more wrong if it was the build date, as that doesn't refer to anything to do with the version. -- thestinger 21:33, 23 January 2015 (UTC)

it doesn't refer to a specific commit

Neither does "r1581.2b039da" refer to a specific commit without the .git repo cmdline interaction, and it is considered best practice to prune $pkgdir of .git:

       find "$pkgdir" -type d -name .git -exec rm -r '{}' +

Can you explain what you mean by "the dates are not always sequential"? The Date standard looks like this:

       git log -1 --format="%cd" --date=short | sed "s|-||g"

It displays the latest commit date from the upstream Git repo.

In the edge case of breaking changes occurring upstream in Git repos on the same day, then package maintainers should increment pkgrel. In practice, "2014-01-01" is much more readable and useful than "r1581.2b039da". The readability benefits of "2014-01-01" vs. "r1581.2b039da" should not be underestimated due to edge cases. --Atweiden (talk) 18:39, 24 January 2015 (UTC)

A git commit id certainly refers to a specific commit. I don't know why you think this has anything to do with shipping the VCS repository in the package. The commit date of the most recent commit may be from 20 weeks ago while the earlier commits were from last week. Git does not have a linear history. It's not going to be changed to use a version format where it can go both backwards and forwards with no way of tracing it back to the commit. Readability is worthless if the information isn't meaningful... There are 2 hard requirements: newer revisions must have higher package versions without exceptions, and it must be possible to trace it back to a commit to properly identify and report bugs. -- thestinger 18:53, 24 January 2015

I am quite certain of what a commit ID looks like, e.g. 2d5749d0315a59f4a8bf73ebbe757b227c8c17a3, not r1581.2b039da.

Can you show me where to find "r1581.2b039da" on a GitHub/BitBucket repo? It doesn't exist anywhere that I can see. You need to have the repo locally, then cd into the Git repo and run:

   git rev-list --count HEAD

That is work-intensive, and it conflicts with the best practice of pruning AUR packages of .git repos.

I previously used the Wiki-recommended Numeric standard, so it's not like I haven't used it. I switched to the Date standard, updating hundreds of packages to reflect it. Why do you suppose that is?

Readability is worthless if the information isn't meaningful

You'd be hard pressed to convince me that the date "2014-01-01" is somehow *less* semantically meaningful or actionable than "r1581". I say this after maintaining for years, hundreds of Git packages in the AUR. You can tell by grepping `pacman -Q` exactly which AUR git packages are out of date with a two-second visit to GitHub. The Date standard is incredibly actionable and pragmatic not to mention simple and immediately understandable by anyone.

newer revisions must have higher package versions without exceptions

Not a problem: "2014-01-01" < "2014-01-02"

it must be possible to trace it back to a commit to properly identify and report bugs

Given how frequently "stable" Go packages and C packages pull in Git dependencies as part of their build process without specifying a certain commit, I disagree with this proposition. Why would recording the version of acmedaemon as 1.5, when it pulls in Git packages without specifying commit ID, be all that different in bug reports than "2014-01-01"? In both cases, you will not be able to track down the exact cause of the bug by commit. In both cases, you will in all likelihood end up consulting the build date. --Atweiden (talk) 03:32, 25 January 2015 (UTC)

I am quite certain of what a commit ID looks like, e.g. 2d5749d0315a59f4a8bf73ebbe757b227c8c17a3, not r1581.2b039da.

In your scenerio, 2b039da is the short version of the commit hash and does point to a specific commit. As for where you find it on Github, it's listed on the commit log, of course. Just look at the right hand side of the page.

You'd be hard pressed to convince me that the date "2014-01-01" is somehow *less* semantically meaningful or actionable than "r1581".

In this case, human readability really is meaningless. Machine readability is all that matters.

Not a problem: "2014-01-01" < "2014-01-02"

And what happens when the commit after that is dated 2013-12-31? As thestinger pointed out, Git is not linear.

Scimmia (talk) 03:40, 25 January 2015 (UTC)

2b039da is the short version of the commit hash

Can this information be easily compared on GitHub to your local package version to check if it is out of date? The answer is no.

It is not actionable, except by cloning the repo yourself, running the Git command, and then manually scanning the Git log for short versions. It's very work-intensive.

Git is not linear

Neither are stable packages on PyPi: https://pypi.python.org/pypi/tackpy/0.9.9a

Pacman views the upgrade of 0.9.9 -> 0.9.9a as a downgrade. See https://aur.archlinux.org/packages/python2-tackpy for a real world example of this edge case. Edge cases should be no reason to prevent readers of the wiki to make informed choices about Git packaging standards. --Atweiden (talk) 04:08, 25 January 2015 (UTC)

Yes, the short hash of each commit is listed in the commit log. Also note that the revision count (the "r" number listed first) is also listed at the top of each repo's page.

As for PyPi's versioning scheme, their choice is different than what Arch's vercmp assumes. The packager needs to take this into account, just as all packagers must take upstream's idiosyncrasies into account. I don't see how it's relevant here.

You also never answered my question, what happens when the last commit is dated before the commit previous to it? This question is critical, as it would cause your versioning scheme to go backwards by any and all measures. That makes it unacceptable.

Scimmia (talk) 04:15, 25 January 2015 (UTC)

Incorrect.

   printf "r%s.%s" "$(git rev-list --count HEAD)" "$(git rev-parse --short HEAD)

The output of the above command does not correspond to any piece of information on GitHub.

If PyPi packagers should take edge cases into account for PyPi packages, then they should do the same for Git packages. Edge cases aren't reason enough to scrub a competing standard from the wiki. --Atweiden (talk) 04:23, 25 January 2015 (UTC)

I'm sorry if you can't find this information on GitHub. It's very, very plainly displayed in the places I already pointed you to. If you would take a minute to check instead of just jumping at me, you might find it.

Scimmia (talk) 04:31, 25 January 2015 (UTC)

the short hash of each commit is listed in the commit log. Also note that the revision count (the "r" number listed first) is also listed at the top of each repo's page.

This is (partially) incorrect. The short hash of each commit is not listed. What is listed in the commit log is the truncated commit ID.

The revision count point I will retract. It didn't appear to work for my own Git repos, but that was because I ran the command from a Git subrepo (edge case :). The date still has the advantage of being more human readable than r1582. If a package has version 2011-10-22, I will know this project is either unmaintained, or it's been a really long time since I last rebuilt it. All without leaving the terminal. --Atweiden (talk) 07:33, 25 January 2015 (UTC)

It's also very easy to look up or link to commits via the abbreviated commit hashes or proper identifiers produced by git describe. -- thestinger 04:33, 25 January 2015 (UTC)

That has nothing to do with non-linear history. It has to do with an incompatibility between Pacman's version comparison and the one chosen by the project. There are ways of dealing with that, but it has nothing to do with this discussion. -- thestinger 04:24, 25 January 2015 (UTC)

The ideal version is the one produced by git describe, because it's easily understood by humans (last tag, commits since last tag, abbreviated commit), is also understood by the various Git commands and GitHub and is monotonically increasing thanks to the inclusion of the last version and revision count since then. The versions it produces (10-7-g8537989) can be directly used with commands like git show. Unlike the build date, it identifies a unique revision of the project, including the state of any submodules.

The only case where another method should be used is when the project doesn't yet have any tagged release or does the releases in a way that's not very sane. The releases should be tagged on master with additional tags in stable branches for fixes backported to old releases. If this isn't the case, the git describe output needs to be emulated. A date is a poor way of doing that, because it's not connected to a specific point in the revision history.

The only case where the commit date would work is if Git is being used as a crippled centralized version control system. If all commits get created against master (no merging branches or patches) in a centralized repository where the system clock is never moved backwards, it could work... but that's not the reality of non-linear, distributed version control.

A build date is even worse, because it only refers to the local state of time and the local state of the repository. Commits from before that timestamp will often be pushed to the repository.

-- thestinger 04:24, 25 January 2015 (UTC)

A packaging standard must be compatible with the lowest common denominator. A minority of Git repos are properly tagged. The Date works as a standard, tags do not.

   git log -1 --format="%cd" --date=short | sed "s|-||g"

The output of the above command shows the date of the most recent commit. If the most recent commit date happens to come before the date of a previous commit, I do understand this is suboptimal, but I consider it an edge case. Even in big projects with contributors in different time zones you will typically see branches merged in with PRs. PRs reset the date to something recent. The linearity of Git is rarely problematic, but I do see where you're coming from (I'm also a happy user of archversion for keeping packages up to date, though archversion works with the Date standard too :) --Atweiden (talk) 04:59, 25 January 2015 (UTC)

The git describe versioning system is the standard defined and used by Git itself. The dashes need to be converted into dots for pkgver and an r may be inserted to make it clearer which part is the revision number. If version-based tags on master are available, this is the only sane way to do it.

Git defaults to fast-forwarding / applying patches without merge commits, as long as there are no conflicts. GitHub will always generate a merge commit but that's not true for anything applied / merged by the developers and then pushed to master. -- thestinger 05:24, 25 January 2015 (UTC)

You've made the claim that most git repos aren't properly tagged, not only now but to me previously. Do you have anything to back that up? IME, the large majority of projects that are in a usable state are either tagged or get tagged as they become stable.

Scimmia (talk) 05:27, 25 January 2015 (UTC)

OK:

   git init

You now have your very own untagged Git repo :).

The majority of Git repos are untagged. If you want to have a standard compatible with the lowest common denominator, by definition you cannot depend on tags being there.

I regularly package Git software in the alpha stages and software maintained by people who are unfamiliar with tagging or aren't bothered to tag. --Atweiden (talk) 07:15, 25 January 2015 (UTC)

As suggested in VCS package guidelines#Fallback, I think the only date that can be used is the build date: for example I've recently used it in a PKGBUILD for a repo without tags or branches, also appending the first 7 characters of the commit hash (the repo is not cloned, so I could only use git-ls-remote). — Kynikos (talk) 03:30, 25 January 2015 (UTC)

Updating a CVS repo

I don't use cvs. How can you describe the pkgver for cvs (for pacman 4.1)?
-- Dracorp (talk) 09:31, 6 April 2013 (UTC)

CVS is not supported in pacman 4.1 like the other VCS tools. You will need to update pkgver manually until CVS support is added.

-- Jstjohn (talk) 22:44, 15 April 2013 (UTC)

Yeah, but how about mentioning that in the article (as well as giving a download example)? Even if it's not that common anymore.

--Det (talk) 22:39, 2 May 2013 (UTC)

The download example can still be found in /usr/share/pacman/. The next version of the ABS package should update it a bit so the download happens in the prepare function where it belongs. As for pkgver, I think the generic example using date covers that, as there's not a way to get a version number from a CVS repo. Maybe a note to that effect?

-- Scimmia (talk) 07:17, 15 May 2013 (UTC)

That makes the most sense, but it might also be a good idea to rename the "Fallback" section to something like "Fallback / CVS" to make it more obvious even when you're just checking out the table of contents.

But as for ABS, as far as I can tell the last commit was over 8 months ago.

--Det (talk) 05:54, 19 May 2013 (UTC)

Hmm, there were a number of patches submitted last month for cleaning up the prototypes, looks like none have been committed yet. I do remember a discussion (IRC maybe?) questioning the proper place for the prototypes, so maybe that's why? Looking at the patches, I was mistaken anyway; they didn't update the darcs or cvs prototypes. Simple enough, I'll send in a patch myself.

--Scimmia (talk) 08:22, 19 May 2013 (UTC)

I use this dirty hack: cvs history -c -a | cut -d' ' -f2 | sort -u | tail -n 1 | sed 's|-||g" ; could probably be improved.

--Buhman (talk) 18:00, 6 June 2013 (UTC)

pkgver function for hg based on tags

I recent came across a way with hg to show the most recent tag, as well as the number of commits from this tag (similar to the output of git describe.)

pkgver() {
  cd local_repo
  hg log -r . --template '{latesttag}.{latesttagdistance}.{node|short}\n'
}

3.0.1.40.ee9a2543fcd6

Please could this be included in the page.

Garyvdm (talk) 09:03, 23 July 2013 (UTC)

shorten hg version

To prevent long package file name, It is proper to use this format

pkgver() {
  cd $_repo
  _id=$(hg identify -i)
  echo $(hg identify -n).${_id:0:4}
}

--Dlin (talk) 05:30, 26 August 2013 (UTC)

fossil

pkgver() {
  cd local_repo
  _id=$(cat manifest.uuid 2>/dev/null)
  echo ${_id:0:4}
}

--Dlin (talk) 05:36, 26 August 2013 (UTC)

set -o pipefail

This change mentioned set -o ... as one of the reasons for reverting, nevertheless one example using set -o pipefail still remains: VCS_package_guidelines#Git (the last example in that section). -- Lahwaacz (talk) 17:41, 6 March 2015 (UTC)

Guidelines on updating version numbers

There are a few version control AUR packages out there which use the correct combined format of release and git commit. However, sometimes the maintainers refuse to update the default pkgver upon new releases of the software, stating that it's unnecessary since it's updated automatically during installation anyway. While this is true, I personally feel it should still be the maintainer's responsibility to update the default pkgver upon actual releases (not individual commits) as otherwise a user will never receive an automatic update and instead has to force-reinstall all(!) such packages regularly.

Should we add this to the guidelines or how do others feel about this?

—This unsigned comment is by Airblader (talk) 09:51, 10 September 2015‎. Please sign your posts with ~~~~!

There's no official update mechanism for AUR packages, so this argument only applies to those using unofficial AUR helpers. Otherwise there's soname bumps; if the package is a library, then pkgver should likely be updated. If it is a library the package depends on, I don't know of a reliable way as pkgrel is stuck to 1 on VCS packages. After all, a soname bump could happen between releases or even commits. -- Alad (talk) 11:00, 10 September 2015 (UTC)

If you're using a VCS package, you should be paying attention to what is happening upstream and choose when you rebuild the package. Managing them the same way as release packages makes no sense.

Scimmia (talk) 13:25, 10 September 2015 (UTC)

Can you explain why it doesn't make sense to manage them the same way? One example is the package i3-git. As of writing this, it's version in the AUR is 4.10.2-xxxxx. The current i3 release is 4.10.4. So why *shouldn't* the package maintainer feel responsible to update it to 4.10.4-xxxxx? It's part of the Arch "feel" to be close to current development of software and I feel that if you maintain a package, you are responsible to update its version even if the actual installation process doesn't change. Sure I can rebuild my git packages, but that's really time intensive. I also cannot monitor all of those packages upstream with reasonable effort. This is exactly what a package management system has version numbers for and keeping them up to date is exactly what package maintainers are for, IMHO. Airblader (talk) 11:39, 11 September 2015 (UTC)

VCS packages will always build the latest revision, regardless of what pkgver on the AUR contains. If a package maintainer bumps pkgver of a VCS package, there is no assurance that the users will actually build the same (working) revision, hence the bumping makes no sense. If you need to track multiple git repos, use something like repocheck to find out if you actually need to rebuild the package. I'm sure there are similar tools for other VCS, or just write your own. -- Lahwaacz (talk) 12:13, 11 September 2015 (UTC)

I don't think that bumping makes no sense just because you build the latest version and not the one stated by pkgver. This is *always* the case, even if the version isn't bumped. The bump isn't meant to build exactly that version, it is meant to trigger an automatic update in the user's package management system that doesn't require checking out the repository and verifying it themselves. Again, IMHO this is what version numbers are for. If we don't care about such packages triggering an update or having any kind of comparable version number, what's the point in even giving them a (meaningful) number? We might as well just set pkgver=1 and be done with it, no? So why all the fancy pkgver() functions for VCS packages? I don't see how the repocheck you linked to is relevant for this. I don't have the repositories of my VCS packages laying around (only in /tmp right after I update them and only until I reboot). I'd like to approach this with a pragmatic argument: I hope we can agree that triggering the update for official releases of the package is some benefit. Given that, what's the *harm* in recommending to update pkgver? If there's an opinion that it makes no sense, that's one thing, but as long as it doesn't harm I think the small benefit sounds like reason enough to me. It's not like updating the pkgver of a package is difficult work. Airblader (talk) 13:20, 11 September 2015 (UTC)

"I hope we can agree that triggering the update for official releases of the package is some benefit." We can't. I really see no benefit there at all. "Given that, what's the *harm* in recommending to update pkgver?" The harm comes from annoying people using those AUR helpers you mentioned. They have correctly chosen when they want to rebuild that package, now their helper is complaining all of the time even though they don't want to rebuild right now. Trigger automatic rebuilds of VCS packages is rarely a good thing, as there are no upsides and real downsides.

Your argument that it is really time intensive to rebuild your VCS packages makes no sense here, as you have to rebuild them either way. If you're unable to monitor upstream, why are you using a VCS package anyway? If you have no idea what's going on, you have no idea what you're getting when you build/rebuild it.

Scimmia (talk) 13:38, 11 September 2015 (UTC)

I don't think that you choose VCS packages because you don't want your package management system to annoy you with updates. If that's something you don't want, then you don't use one at all (why would I choose to not receive updates for *that one* package, but for all 500 others?). IMHO, you choose VCS packages because you want to be close to current development, either "just because" or because you need(ed) it for something else. Also, sometimes there just is *only* the VCS version and no package for the release. Also, if you don't want to receive updates about a certain package, you can always ignore it in the config. I don't think it's good to deny *all* users the notification of updates just because *some* might not want them.

I also can't agree with "triggering automatic rebuilds of VCS packages is rarely a good thing". I never see upront what I actually build, so there's always that "risk". You don't install VCS packages if you want the release version. Furthermore, in practice, it seems rare that rebuilding a VCS package causes any issues since they usually point to the master branches which in most projects have some QA before merging.

Your point about having to rebuild them anyway and hence not saving time is not correct. You're arguing that I should just rebuild all VCS packages all the time just in case there was an update. I'm suggesting the information whether a package needs to be rebuilt is transported in the pkgver. This saves a *drastic* amount of time as I'm not speculating. Of course I can check the Github page of each package's upstream URL before rebuilding it, but this puts the work of versioning the package onto each and every user rather than the package maintainer. Again, if this is the desired behavior, we should recommend that VCS packages just set pkgver=1. I don't see why they need to provide any version info if it's completely unnecessary that it ever be updated. Airblader (talk) 13:48, 11 September 2015 (UTC)

"I'm suggesting the information whether a package needs to be rebuilt is transported in the pkgver." This is really the heart of the matter. This is completely and totally wrong for VCS packages. Whether the package needs a rebuild is up to your judgement, not the pkgver.

There's nothing wrong with setting pkgver=1 (unless the actual version is less than 1). Some AUR package already do this, using "LATEST". The important thing is that the correct pkgver appears on the build package.

Scimmia (talk) 13:59, 11 September 2015 (UTC)

I just don't agree that it's "completely and totally" wrong. Yeah, sure, I can always decide to rebuild it myself. No one is taking this option away from anybody. But setting the pkgver when the actual release changes means the user is notified of this without having to care about it themselves (all n of them). Anyway, I don't think this is going anywhere. To me, letting the pkgver rot destroys a major point to why I use a package manager in the first place and makes me wonder why that person wants to be a package maintainer if they don't maintain the package, but clearly there's very different views on this. Thanks for the discussion anyway, at least I know now that there's no common view on it so there's no point in pushing for a change of guideline. Airblader (talk) 22:58, 11 September 2015 (UTC)

Sorry for reopening, I didn't make it in time to reply earlier...

I'll start by saying that I agree that VCS package maintainers shouldn't bump pkgver manually. However, suggesting to use a less confusing pkgvar value would seem sensible to me, e.g. we could indeed give "LATEST" as an example.

— Kynikos (talk) 04:04, 13 September 2015 (UTC)

I also wanted to try to give @Airblader another perspective. You have mentioned 2 different cases:

• A package has both a "stable" and a "vcs" version, e.g. i3 and i3-git^AUR: why are you using i3-git^AUR in this case if you want to follow the normal upstream releases?
• A package only has a "vcs" version, e.g. wmfs2-git^AUR (dead, I know, but take it as an example): in this case the AUR maintainer can't be blamed for not bumping pkgver manually, so the problem is actually that there's nobody maintaining a "stable" PKGBUILD, and you could be the one filling that gap!

— Kynikos (talk) 04:04, 13 September 2015 (UTC)

Sorry for the necroposting, but what about AUR packages that have explicitly as a dependency a VCS package (even if the stable version exists), or more commonly packages for which upstream doesn't bother to make discrete releases?

ThePirate42 (talk) 18:20, 21 January 2022 (UTC)

What about them? Scimmia (talk) 18:26, 21 January 2022 (UTC)

Well, since sometime there isn't any choice other than using a VCS package, I feel like there should be some way (not necessarily the pkgver) for PKGBUILD mantainers to, at least, signal that some important change has happened upstream; and this in a way that is inspectable by automated means. Of course this should be optional for mantainers, but it seems to me that it could save a lot of time for people that need a VCS package, but would otherwise be perfectly happy to just interact with non VCS AUR pages that can be easily parsed by scripts and not with various, uneven upstream pages.

ThePirate42 (talk) 23:14, 21 January 2022 (UTC)

Add information about sub-modules in GIT section

Git has submodules that can be updated rather than having to download the entire git repo for an update. Someone shared this with me: http://sprunge.us/HKdJ?md and I think it could be useful to document this sort of thing in the wiki. I'd try to myself but I'm not super familiar with git submodules.Meskarune (talk) 23:35, 8 December 2016 (UTC)

That's what https://wiki.archlinux.org/index.php/VCS_package_guidelines#Git_Submodules already says, isn't it?

Scimmia (talk) 00:14, 9 December 2016 (UTC)

I was thinking it should be in the git section and expanded with a better explanation or at the very least links to more information on git sub-modules. Also why someone would want to even mess with sub-modules. I just don't feel like I know enough to write something further up myself. Meskarune (talk) 01:54, 9 December 2016 (UTC)

Git submodules: sometimes not all submodules are needed

As per this writeup by Earnestly (which I understand is the initial source of that section): https://ptpb.pw/r/N6-z.md Some packages shouldn't try to checkout all submodules, because it is wasteful and unneeded. It might be nice to expand on that section by mentioning this. -- Eschwartz (talk) 14:19, 12 July 2017 (UTC)

Add notice to this page?

The Arch User Repository page indicates:

Note: VCS packages are not considered out of date when the pkgver changes, so please do not flag them as the maintainer will merely unflag the package and ignore you. AUR maintainers should not commit mere pkgver bumps.

Perhaps this should be included on this page as well, to inform new maintainers that they should not be making pkgver bumps for these packages. That being said, there should probably be a way to update the pkgver shown by the AUR web UI, or disable it entirely for VCS packages, in order to prevent others from believing a package is out of date and flagging it. JacobHenner (talk) 11:33, 17 November 2017 (UTC)

pacman 4.1

Shall we remove the references to 4.1? What was possible / the practice in the past may help to understand certain outdated packages but 5 years starts to be very long and we should turn the page at one stage I suppose. This is the impression of a newcomer who wants to get to the point and is confused by the reference to such old versions but don't hesitate to correct me if this is still important. Kewl (talk) 10:29, 14 January 2018 (UTC)

Agree. -- Blackteahamburger (talk) 15:00, 30 June 2020 (UTC)

Suggest to use https instead of git protocol when possible

A lot of public structures/companies block the default git port (9418). This problem is explained here : https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols#_the_cons_4 Using git+https:// urls for sources is supported by most (if not all) git servers, and https will most likely not be blocked.

—This unsigned comment is by Salamandar (talk) 10:39, 22 October 2018‎. Please sign your posts with ~~~~!

+1, I'd also mention that it's a security issue because git:// and http:// aren't encrypted and vulnerable to MITM attacks. --Larivact (talk) 19:10, 22 October 2018 (UTC)

+1, That's gold for me, as I'm normally behind a proxy that blocks git://, but allows git+https://... -- Josephgbr (talk) 02:19, 16 April 2019 (UTC)

I see this is done in "An example Git source array". How about the other two places that illustrate using git sources? (left of vcs+ bullet point in VCS sources, on two lines in the Git submodules tip) quequotion (talk) 15:18, 5 June 2019 (UTC)

GitHub API usage

Some git repos are really huge (ex. QGIS) and git-clone process takes a lot of time and space. So, here is an example how to use GH API to obtain pkgver:

_orgname=foo
_pkgname=bar
_branch=master
_gh_api_url="https://api.github.com/repos/${_orgname}/${_pkgname}"
...
makedepends=('jq')
...
source=("https://github.com/${_orgname}/${_pkgname}/archive/${_branch}.tar.gz")
sha256sums=('SKIP')

pkgver() {
  read -r tag_name tag_sha <<<$(curl -s "${_gh_api_url}/tags" | \
    jq -r '.[0]|[.name,.commit.sha]|@sh' | sed "s/'//g")
  head=$(curl -s "${_gh_api_url}/git/refs/heads/${_branch}" | \
    jq -r '.object.sha')
  count=$(curl -s "${_gh_api_url}/compare/${tag_sha}...${head}" | \
    jq -r '.total_commits')

  printf "%s.r%s.g%.8s" "${tag_name}" "${count}" "${head}"
}

0.1.r7.g47de1a51

—This unsigned comment is by Sikmir (talk) 2019-06-28. Please sign your posts with ~~~~!

makepkg still requires cloning the whole source code repository to build the package. In fact, the source code fetching happens before the pkgver function. IMO, this option does not seem to have any use for packaging in Arch. By the way, if you use the binary jq, make sure to add jq to makedepends array. -- Josephgbr (talk) 11:37, 28 June 2019 (UTC)

No! It doesn't require cloning, see source variable, there is only gzipped snapshot. Sikmir (talk) 16:18, 28 June 2019 (UTC)

I didn't see the tarball in source before, but in my opinion a PKGBUILD with a branch/tag's tarball as source does not classify as a VCS package and, therefore, should not be included to this page. I understand GitHub's snapshot tarball represents the last commit of the software, but still you are not using a VCS. Personally, since Arch Linux has these well defined type of packages, normal/non-VCS package with defined version and VCS packages with dynamic version, I would make my dynamic versioning packages as VCS package due to the advantage of just updating the source code repository instead of re-downloading it all. -- Josephgbr (talk) 18:08, 28 June 2019 (UTC)

Add GPG Signature verification note

It is possible to verify the git sources using gpg. This is documented in "man PKGBUILD", but most people do not know this. It would be easy to have this information in the tip section of this guideline.

source=(<url>?signed)

validpgpkeys=('<keyid>')

NicoHood (talk) 07:54, 3 June 2020 (UTC)

Manually constructed 'git describe' output should be forward-compatible

The recommendation for when no tags exist isn't forward-compatible wrt adding tags -- note the absence of a 'g' prefix in the SHA.

Gesh (talk) 15:22, 20 December 2020 (UTC)

It could be added for consistency, but it really doesn't matter. The versions are incremented by the revision, and if a tag gets added to the front of that, that's obviously an upgrade. Functionally, nothing would change.

Scimmia (talk) 15:26, 20 December 2020 (UTC)

This inconsistency bugs me too. However, I'd rather remove the g from tagged versions instead of adding it to untagged versions. That would be consistent with the suggestions for mercurial. Also, commands like git log 1596c1f do not accept the g prefix. So removing the prefix seems only logical.

In all sed commands, change s/\([^-]*-g\)/r\1/ to s/\([^-]*-\)g/r\1/.

Also; I'd suggest changing the combined function as follows. This does not require changing the settings to set -o pipefail and keeps both versioning alternatives consistent with a single change of the sed command.

pkgver() {
  cd "srandom"
  {
    git describe --long --tags 2>/dev/null ||
    echo "$(git rev-list --count HEAD)-g$(git describe --always)"
  } | sed 's/\([^-]*-\)g/r\1/;s/-/./g'
}

Schaetzc (talk) 14:23, 10 September 2021 (UTC)

git: core.abbrev results in non-deterministic git-describe output

The Git guidelines suggests using "git describe --long". However the output of this command is affected by the "core.abbrev" Git config. The kernel guidelines recommend changing the value of this config option. As a result, 12 characters of the commit hash will be included, instead of 8. This causes the package to always be marked out-of-date for other users.

To fix this, could we add "--abbrev=12" top the suggested Git command?

—This unsigned comment is by Emersion (talk) 2022-09-28T19:41:50. Please sign your posts with ~~~~!

Maybe the suggested git command should have "--abbrev=8" instead of 12 so that existing package versions don't wildly change after this change. (Generally packages should IMO only change in a way that does not conflict with historic package versions)

Ferdi265 (talk) 12:51, 29 September 2022 (UTC+2)

The default is 7 actually. The option is not defined by default, but according to git-describe(1):

will vary according to the number of objects in the repository with a default of 7

Hanabishi (talk) 14:33, 10 February 2023 (UTC)

Guidelines: (mostly) cosmetic improvements

Continuing this discussion, after the structural changes were finished, I made a separate edit for the cosmetic and technical changes that had been proposed alongside them, which has been reverted.

Let's discuss the problems with the content as it stands and how to make it better.

As you can see from the diff linked above, the changes break down to roughly six sections.

The first is a simple substitution of pkgname with pkgbase.

Problem: pkgname doesn't cover PKGBUILDs for split packages.

Solution: pkgbase covers both single and split pacakges.

The second is a simplification of what is to be done about pkgrel and pkgver.

Problem: The list of specific things that warrant a change to pkgver or pkgrel is confusing, inherently incomplete, and long.

Solution: Maintainers should be directed to prioritize updating pkgrel whenever the PKGBUILD itself has changed. If we are to tell them to update pkgver at all, we need to be clear and concise about when that is. Last I checked, consenus was that it is never really necessary to update pkgver for a VCS package.

The third regards conflicts and provides.

Problem: The pronoun "what" is unclear, while the example indicates that the objective of this section is to inform maintainers that the repository version of the package is to be included to these arrays.

Solution: Explicitly state that the repository version of the package needs to be appended to these arrays.

The fourth regards replaces.

Problem: This is a cryptic warning that doesn't communicate anything about what replaces is for or what kind of problems using it could cause, which will almost certainly result in people getting confused and trying it out anyway

Solution: Explain what replaces actually does and when it might be useful. An alternative approach might be to move the clarification to PKGBUILD#replaces (using the word "deprecate" somewhere) and link there with a shorter, lest cryptic statment or remove this section altogether (we don't really need to say "see this? don't touch it." when we could just ignore it to begin with).

The fifth is a totally cosmetic edit, changing makedepends=() to makedepends.

Problem: foobar=(), as a convention is ugly, unnecessary in this context, and inconsistent with other pages in the wiki.

Solution: foobar is sufficent and easier to process.

The sixth regards skipping checksums for VCS sources.

Problem: md5sums=() does not account for the various kinds of checksums that may be used and simply "adding" 'SKIP' may place it in the wrong order if a VCS package has multiple sources.

Solution: use a wildcard to indicate that there are various kinds of checksums, and be more technical about what to do with 'SKIP'.

Please see the diff linked above if you'd like to review my changes that were reverted, or see them on the full page draft made for the previous discussion. quequotion (talk) 15:23, 11 January 2023 (UTC)

For the first, pkgname *does* cover both split and non-split packages. If it's specified separately, it doesn't really matter, the pkgname is what's important here. This change is completely wrong.

For the second, no, updating the pkgrel when there's a newer pkgver makes no sense at all. It won't be picked up by anything checking for updates, making it pretty useless. The consensus is to not *just* bump the pkgver, bumping it when changing things in the PKGBUILD is normal.

For the third, this could work, but it still needs to be clear that all conflicts and provides need to be included, not just the non-vcs version.

For the fourth, this could be done much more simply by just linking to the PKGBUILD page, but leaving what we already have here. The point is to stop people from saying it replaces the non-vcs version, which people did a lot before this warning was put here.

The fifth and sixth are something for the wiki maintainers who put together the style guidelines to address.

Scimmia (talk) 15:40, 11 January 2023 (UTC)

>If it's specified separately

This invites someone to forget to append their -vcs to one of the separate package routines; appending it to pkgbase ensures it gets appended to all of them.

>updating the pkgrel when there's a newer pkgver makes no sense

In the context of VCS packages, how often is updating pkgver really relevant? There's a tip immediately following this suggesting maintainers use --holdver. On top of that, can we actually account for how "anything checking for updates" works on the wiki? That seems dangerously close to legitimizing AUR helpers. This has been a contentious issue on this page for a long time; it would be helpful to have a clear policy statement here.

For the third, how about:

• If available, append the repository version of the package to conflicts and provides (e.g. for fluxbox-git^AUR: conflicts=('fluxbox') and provides=('fluxbox')).

For the fourth, how about:

• Do not append the repository version of the package to replaces, which is only for deprecation.

quequotion (talk) 17:31, 11 January 2023 (UTC)

1. The package function names are based on the pkgname, not the pkgbase, so your reasoning here doesn't make any sense to me.

2. Bumping the pkgrel all of the time with an old pkgver is useless in all cases. Period.

3. It's not just about repository packages, and it's not always just <pkgname> vs <pkgname>-git. There can be various different builds, which is why the wording that's there is generic but adds the example to a common case.

4. Also not just about repo packages, although that's where it causes most issues.

Scimmia (talk) 16:51, 13 January 2023 (UTC)

1. I was thinking pkgname derives from pkgbase when defined; apparently the other way around. OK.

2. That's totally opposed to what has been policy as laid out on this page and argued on this talk page since long before I ever used Archlinux. The point with VCS packages is that pkgver does not matter *in the AUR* most of the time: users of the package should be keeping track of upstream changes on their own, while it's maintainer should only update when changes to the PKGBUILD are required, ie changes that denote a new pkgver. VCS package maintainers are generally supposed to ignore pkgver unless they have a good reason to update it.

3 & 4 True, but the wider scope of these arrays should be handled on the PKGBUILD page. These blurbs, being on this page, are specifically for the context of VCS packages. ie, their intent is to instruct maintainers what additional considerations there are for VCS packages, which happens to be the interaction with repository versions of the packages.

quequotion (talk) 00:45, 16 January 2023 (UTC)

About git submodules in combination with `$SRCDEST`

As it stands there are a lot of projects that use submodules to pull in sources, and sometimes it just not possible to depend on system libraries, either because the build configuration upstream is using is not capable, it requires extensive patching etc. In any case, you can find yourself in two particular situations.

1. The first one is using a fork of the original project as a submodule. This can cause issues with the shared SRCDEST option for makepkg the remotes will be different and pulling it in source array will complain about it.
2. The second one is the program being built renames the submodule directory into something different than the repository name or uses a sub-directory. In this case renaming the repository in the source array can cause multiples of the same git repository in $SRCDEST.

Ideally I would like to see two additions to the guidelines.

1. One is that if the submodule pulls in from a fork, the source array should clone into a $pkgname specific directory. I think a format like <repo_name>-$pkgname::git+https://<source_url>.git would suffice. Example: xz-pcsx2::git+https://github.com/PCSX2/xz.git
2. The other one would be to add something along the lines of the following in tips and tricks

   _submodules=(
       xz-pcsx2::3rdparty/xz/xz
       googletest::3rdparty/gtest
       Vulkan-Headers::3rdparty/vulkan-headers
       rcheevos::3rdparty/rcheevos/rcheevos
   )
   for submodule in ${_pcsx2_submodules[@]}; do
       git submodule init "${submodule#*::}"
       git submodule set-url "${submodule#*::}" "$srcdir"/"${submodule%::*}"
       git -c protocol.file.allow=always submodule update "${submodule#*::}"
   done

The snippet above follows the same pattern as the `source` array or splitting the name of the source and the path of the submodule with a double colon `::`. The first part is the name as specified in the source array, the second part is the path of the submodule in relative to the directory of program being built.. Example source array:

source=(
   git+https://github.com/PCSX2/pcsx2.git
   xz-pcsx2::git+https://github.com/PCSX2/xz.git
   git+https://github.com/google/googletest.git
   git+https://github.com/KhronosGroup/Vulkan-Headers.git
   git+https://github.com/RetroAchievements/rcheevos.git
)

--LoathingKernel (talk) 14:15, 25 March 2023 (UTC)

Honestly, I think a single git submodule update --init --recursive is much more convinient than doing all that stuff manually. I don't know why it is not even mentioned in the article.

Using it, you only need 1 source - main repository. Submodules are resolved and pulled automatically. It works like a charm, at least with GitHub.

And it perfectly solves the described problem, because in this case submodules are stored in-place inside the main repo.

Hanabishi (talk) 15:25, 25 March 2023 (UTC)

Using git submodule update --init --recursive breaks both the convention of $SRCDEST for shared sources handling and makepkg --allsource.

It also creates unnecessary network traffic, server load and time waste because the whole git tree has to be fetched for repeated builds. Using --depth 1 can help with these but it still breaks makepkg functionality.

I am aware of other build system, such as rust's cargo or go's also break this convention, but that is probably a much larger discussion than this one.

In my opinion there shouldn't be any network access in prepare()

LoathingKernel (talk) 15:50, 25 March 2023 (UTC)

Using --filter=tree:0 mitigates the traffic issue almost completely. (Btw it is also very useful for VCS packages in general, too bad makepkg's GITFLAGS varable is still not in the stable release.)

Regaring the convention, IMO complying with it simply does not worth such amount of headache that the manual method brings. And considering the problem you described, it turns into a major PITA.

Hanabishi (talk) 16:08, 25 March 2023 (UTC)

I see what you mean with --filter-tree:0, that's a good alternative.

For the sake of completeness I want to mention another problematic case, which should also be mentioned. makepkg clones from the local repo making remote a reference to the locally cloned bare repository. This will make submodules specified using relative paths fail. Maybe this workaround should also be mentioned somewhere.

git remote set-url origin https://github.com/RPCS3/rpcs3.git

git -c protocol.file.allow=always submodule update --init --depth=1

LoathingKernel (talk) 18:16, 25 March 2023 (UTC)

Yeah, that's kind of a drawback. But every particular package can be tuned individually.

Also there may be other good ways. I personally not very deep into git documentation.

Hanabishi (talk) 18:49, 25 March 2023 (UTC)

I almost wonder if alot of this could be avoided by giving makepkg / the build environments a way to parse .gitmodules if it exists in a project

The advantage being that it would auto handle any odd naming conventions and see recursive modules.

Weirdbeard (talk) 17:24 25 March 2023 (UTC)

Naming convention

How is the suffix determined for other version control systems? e.g. fossil, pijul, dat ... Do we just use the full name of the VCS?

Eclairevoyant (talk) 11:21, 27 June 2023 (UTC)

pkgver for packages with submodules

Does it make sense to include the submodule's version to the pkgver? Example package is sdl-jstest. If submodule updates, and you rebuild the package, it will have the same pkgver, but actually it will be updated package. Ashark (talk) 16:57, 10 September 2023 (UTC)

For the submodules the exact version (commit) that is built is pinned by the parent repository. Unless the packager explicitly updates the submodule in the PKGBUILD to a different commit, the same version of the submodule will be built, despite updates in the submodule's origin repository. If left unchanged, the submodules will build the version the parent repository has pinned and if parent repo updates the submodule, the version will also change as the action of updating them is a revision (this is relevant to VCS packages). Your example from the AUR has the parent repository pinned to a specific commit, meaning it will always build the same version of the software and its submodules.

LoathingKernel (talk) 09:56, 11 September 2023 (UTC)

Thanks for clarification! Ashark (talk) 10:00, 11 September 2023 (UTC)