Talk:AUR helpers

"Reference" implementation

This is an alternative to #Reliable Updater. Instead of an arbitrary set of test packages, we could write up a "specification" on what a reliable AUR helper should do. This should also be more helpful for potential AUR helper writers who otherwise have to wade through complex, fully-featured AUR helpers.

I propose a minimal reference implementation with the following points:

• No client-side workarounds for upstream limitations. In particular, a reference implementation does not need to score full points on split packages, as makepkg --pkg was removed with pacman 5.
• Minimal language constructs in e.g. a scripting language like dash.
• Prefer simplicity of implementation over being fully featured. In particular, an implementation may only support git clone and not git diff.

My initial plan was to keep such an implementation in a man page aurhelper(7) (hosted as part of aurutils), but we can consider including on a sub-page of this article. It could be then linked from the comparison table. Thoughts? -- Alad (talk) 13:28, 8 March 2018 (UTC)

Generally agree with the idea, but I don't think there is a way around a set of PKGBUILDs that could be used to test helpers in a local AUR instance. F.e., I wouldn't define a "reliable" helper that doesn't handle split packages well. Since helpers are tolerated rather than supported, upstream limitations of the AUR might be temporary or permanent, meaning the limitation would actually be in the helper itself (f.e. like regex support). Also, I'd use pseudo code for such a reference as the actual implementation itself doesn't matter, unless you'd like to write a new minimalist helper. Spyhawk (talk) 15:26, 8 March 2018 (UTC)

Apart from FS#56602, I can't think of a case where upstream opposed removing limitations, even if helpers directly benefited. cf. the regex support discussed in [1] or the exit codes finally introduced in makepkg 5.1 which made automatic building significantly easier imo. To me it seems that the main reason we have these AUR limations is due to the minimal interest of helper writers in contributing upstream, and upstream itself having different priorities. Not sure why former is the case, the PHP codebase may play part in it - at least it does for me.

You can keep dash close enough to pseudo-code, I guess less so if you want a complete example rather than exemplary code blocks. For the PKGBUILD set, I use this: [2] -- Alad (talk) 18:34, 8 March 2018 (UTC)

My understanding is that changes that aren't invasive will be accepted upstream, but otherwise might be rejected (see [3]). One prominent example that comes to mind is FS#48796. It's not really relevant anymore since x86 has been officially dropped, but the solution would involve duplicating DB tables on the server, which isn't trivial to implement/migrate. Many of the feature requests involve non-trivial code change, which is the main reason nobody pushed patches; I dislike PHP but the language itself isn't too hard either. For regex, see the bottom of [4], which is the follow-up of your link above.

Your testsuite seems interesting (thanks for the link), but one advantage of having a fixed set of packages is that these packages might be updated and change, making these edge cases difficult to test. This happened quite a few times with my own list of test packages in the past and this was rather annoying. Spyhawk (talk) 20:20, 8 March 2018 (UTC)

Expand Secure criteria to include other (non-PKGBUILD) bundled files

[5], in particular [6]

The new criteria would be as follows:

• PKGBUILD, no other files -> Partial
• Other subset of files that includes the PKGBUILD -> Partial
• No PKGBUILD -> No
• All files in the git repo or tar archive -> Yes

Similar to the Diff view column. -- Alad (talk) 16:32, 4 July 2018 (UTC)

good idea, you also mentioned this for aurman a few months ago, see: https://github.com/polygamma/aurman/issues/25#issuecomment-371971155 really a good idea to implement it in a way, so that changes of all known files are being shown Polygamma (talk) 17:07, 4 July 2018 (UTC)

"All files in the git repo or tar archive -> Yes" What exactly do you mean by all files? Build files often contain non text files such as images. Git diff is smart enough to hide these but then you could consider that partial because not all files are covered.

In my opinion all a helper has to do to be secure it pause and allow the user to read the build files. The helper does not even need to offer to open them for you that's the user's responsibility. Anything more than that is nice to have but not strictly needed. Morganamilo (talk) 20:25, 4 July 2018 (UTC)

If this qualifies as "nice to have", there has to be an explicit warning that a green entry in the "Secure" column does not cover other files, files which may cause more harm than the PKGBUILD itself (such as .install files or exectuables called from the PKGBUILD). In either case it's misleading, since you either give the impression that viewing PKGBUILDs alone is sufficient (with the current criteria), or include a warning that diminguishes the value of the criteria in the first place.

Latter is similar to "Native pacman", in that you have a warning at the article top warning against any sort of pacman wrapping, and criteria in the table that ignore this warning, or even reward behavior which goes against it. -- Alad (talk) 17:07, 8 July 2018 (UTC)

That's a fair point, what about changing the name to "show files before sourcing" or something? Seems more accurate. Then it would make sense that not showing .install files to be partial. The only problem I see that it's not as hard hitting as "secure". Morganamilo (talk) 20:11, 8 July 2018 (UTC)

It cuts both ways: it's an effective deterrent against broken helpers, but it also gives the impression that using a "Secure" helper makes usage of the AUR safe, which it definitely doesn't. I'm not sure on what different name to use, though. -- Alad (talk) 17:25, 14 July 2018 (UTC)

I guess "File view" could work. -- Alad (talk) 17:44, 14 July 2018 (UTC)

The column name was updated to "File review". Are there remaining helpers that only display the PKGBUILD? (trizen^AUR springs to mind) -- Alad (talk) 15:30, 23 August 2018 (UTC)

Pacman wrappers Specificity

I wanted to introduce the manual AUR package selection specificity for pacman wrappers. For example yay and pakku only allow -Syu where they update system packages and AUR packages at the same time. pikaur when doing -Syu also updates system packages and AUR packages at the same time but prompt user for manual selection for AUR packages, because for example I want to update to the new version of X program but not the Y lib (AUR) that is used only by Z program (AUR) so until Z is not updated I don't want to update Y. And yaourt when doing -Syu updates only system packages, -Syu --aur is needed for updating AUR packages as well, and a manual selection for AUR packages is available. That's why I wanted to introduce the manual AUR package selection specificity. -- Noraj (talk) 11:16, 9 September 2018 (UTC)

Why just not specify --ignore ylib? It is problem with zprogram if it does not use versioned deps and allows library update that breaks it. Helper should just fail to satisfy AUR deps in correct case, bail out and user should specify --ignore manually explicitly IMO.

Additionally yay asks you what AUR packages to not upgrade that also provides solution to this XY problem.

All-in-all if you can no longer build some package from AUR correctly because of dependency update that package is broken and should be fixed or be removed instead. And helpers should ideally not provide workarounds for problems that broken package introduce.

-- Svito (talk) 13:06, 9 September 2018 (UTC)

It's not really about broken programs. It could just as easily apply to some package that you're sure will compile fine, but you also remember takes 45 minutes to build, which you don't want to spend right now.

Also I really don't consider using versioned deps, to be the correct case. -- Eschwartz (talk) 14:45, 9 September 2018 (UTC)

That's exactly my use case. Whether to allow it as a specificity or not? I'd probably say sure. But I also think there should be a cap on how many specificities an entry may have. I'd say max 4 or 5, otherwise the column may as well be called feature list. Morganamilo (talk) 14:57, 9 September 2018 (UTC)

New test cases for dependency resolution

ros-foo-desktop-meta have always been difficult to build, even more so with KDE4 libs moved to AUR (see arch-dev-public). Besides the sheer number of dependencies, they otherwise have little interesting properties either.

I propose to instead use various cross-compilation packages as test cases, e.g. mingw-w64-zlib^AUR and powerpc-linux-gnu-gcc^AUR. These appear very efficient at exposing problems with complex dependency algorithms (see for example [7] or aurman's issues with nsis^AUR) and don't take 2 years to build either. -- Alad (talk) 11:34, 14 September 2018 (UTC)

We could also add some simpler cases, like fortune-mod-all-en^AUR, and add details similar to the Split packages description. That way, all existing entries with "Yes" in Reliable solver would at minimum have "Partial". -- Alad (talk) 12:01, 14 September 2018 (UTC)

I believe the mingw stuff has a bunch of circular dependencies and a bootstrapping process. Do you think AUR helper's should be expected to handle this? Morganamilo (talk) 15:25, 14 September 2018 (UTC)

No, I don't think so. There will always be cases that can only be dealt reasonably manually, simply because the involved complexity in implementation isn't worth it. See also related sicussion Talk:AUR_helpers#.22Reference.22_implementation above, where a set of fixed reference packages would be better than live packages. No idea how to dealt with than without a local AUR instance though. -Spyhawk (talk) 10:42, 15 September 2018 (UTC)

The mingw packages haven't had cycles in (global) depends for a while. As such, makepkg -r works fine for these packages without manual intervention or "bootstrapping". If helpers fail, it may be because of handling split depends contrary to PKGBUILD(5), or other flaws in their dependency algorithms. -- Alad (talk) 10:05, 18 September 2018 (UTC)

Erroneous implication that AUR helpers are supported

In Special:Diff/544344 the meaning of the warning was totally stripped away. By longstanding community tradition, AUR helpers are not supported, no ifs ands or buts. It has nothing to do with the rule against AUR helpers in the official repos -- the two concepts merely coincide. If anything, I'd argue that the link in question should be removed -- it seems almost entirely offtopic, and one developer assertion of this fact does not really add more corroborating proof than simply saying so in the wiki I guess. -- Eschwartz (talk) 07:07, 28 September 2018 (UTC)

Removed link [8], closing. -- Alad (talk) 12:08, 1 October 2018 (UTC)

Outsourcing to GitHub

To put an end to these unannounced edits taking excessive wiki admin time, I will be moving the comparison table to github [9], and the article remainders to the (more closely followed) AUR article. Besides the pull request/review model, there are some other benefits to this approach, such as keeping AUR helper discussions in one place. These discussions have often taken place on github before, often in unrelated/offtopic issues for helper projects.

I've invited most of the regular contributors to the AUR helpers article and related discussion to the new "organization". -- Alad (talk) 12:05, 1 October 2018 (UTC)

I disagree with the outsourcing to GitHub. Can't the page be locked to administrators? The fact that MediaWiki's contribution model is limited affects all pages, not just this one, and just because discussions take place somewhere doesn't mean documentation should move there too. --Larivact (talk) 16:28, 1 October 2018 (UTC)

I think existing editors do a good job of maintaining this article in good shape and reverting bad edits. I made edit in question at late night and it was reverted in the morning and it is not like I started edit war or disagree with revert reason. I think revert was good and started a discussion that already resulted in later improvement to the article. -- Svito (talk) 16:44, 1 October 2018 (UTC)

Yeah but Alad's point is that it takes excessive wiki admin time. --Larivact (talk) 16:46, 1 October 2018 (UTC)

I can do my part and stop authoring edits on this and other articles and go back to only patrolling and this problem would mostly disappear. -- Svito (talk) 17:15, 1 October 2018 (UTC)

+1 for locking. If Alad is afraid of his personal time (since he's the main admin working on that article afaik), moving it to GH will only take more of his free time there :) --Spyhawk (talk) 20:20, 1 October 2018 (UTC)

Locking to administrators is something I have tried and which was not fruitful in the long term. It put all editing responsibility on my behalf and was unfair to users who contributed fairly to the article. The note on not editing is equally useless when even official wiki maintainers disregard it.

Now as it turns out, while github has a useful permission model, duplicating mediawiki in markdown is horrifying. GitHub wikis can't be used with pull requests either. I will investigate if additional user groups can be added to the wiki, something I've wanted to for a while. If that doesn't work out, it's perhaps time to leave the AUR helper chapter behind for good. It's caused enough trouble and the majority of the target audience does not understand the content anyway. -- Alad (talk) 08:18, 5 October 2018 (UTC)

I understand your worries, since this topic is a black hole that sucks an enormous amount of time and energy. I do have doubt about moving it externally since this wouldn't solve any of these problems, especially if you take care of it by yourself. I would suggest to revert the article to its initial form: a simple alphabetical list of helpers available without any comparison table. Don't externalise the info, just remove it - and put a restriction on creating such a comparison in the official wiki. Users should make their mind by themselves, and if some other people want to create a fair (or biased) comparison page somewhere else, so be it. --Spyhawk (talk) 12:24, 5 October 2018 (UTC)

Comparison tables help users make up their minds, an alphabetical list would be useless. I doubt a fair comparison page would be created elsewhere. --Larivact (talk) 13:05, 5 October 2018 (UTC)

Except that it has proven over time that the table doesn't help in making an informed decision - as you might expect with any other article or table in the wiki - but rather lends excessive bias to whatever entry happens to have the greatest amount of green colored cells, resp. is placed at the top of the table.

Regarding an alphabetical list, I've considered properly using the aur keyword instead. It requires no maintenance on our behalf and should always be updated. There are a few false positives, because no exact match can be done on "aur", e.g. "thesaurus" is included. [10] -- Alad (talk) 11:06, 8 October 2018 (UTC)