Difference between revisions of "Talk:AUR helpers"

From ArchWiki
Jump to: navigation, search
(Secure column in comparaison table)
(Comparison table - solver: new test-case: remove details)
 
(155 intermediate revisions by 8 users not shown)
Line 1: Line 1:
Authors of each front end should post a short (2-3 line) description of their creation, along with a homepage link and an AUR link (where applicable). A link to a screenshot page would also be nice (if applicable).
+
{{Note|'''Moderation''' — If your AUR helper does [[partial upgrade]]s ''without explicit user intervention'' (i.e, specifying {{ic|-Sy}} on the command line), it has no place on this page or anywhere else on ArchWiki. No exceptions. -- [[User:Alad|Alad]] ([[User talk:Alad|talk]]) 09:37, 20 September 2015 (UTC)
 +
}}
  
== Secure column in comparaison table ==
+
== Comparison table - build directory ==
  
Description says "tries to protect the user", I don't know what "tries" means but if we take the default behavior of aur helpers marked as secure :
+
Considering /tmp is mounted as tmpfs on Arch, and the potential downsides from building in RAM (running out of space), I think a column with the default build location for various helpers would be helpful.  
*owl remains on cower to download deps so, it doesn't source PKGBUILD but calls makepkg without further questions, so finally, PKGBUILD is sourced.
+
*aura does the same
+
*pbfetch sources PKGBUILD (even if it removes build ())
+
*pacaur sources PKGBUILD (it can be configured to remains on cower)
+
...
+
  
As far as I know, only cower is secure (it builds/installs nothing) and spinach (and pacaur with secure on) ask before calling makepkg.
+
The default values I've garnered so far, assuming TMPDIR is not set:
  
The only thing secure in dealing with AUR package is knowing what AUR is about.
+
* aurutils: $XDG_CACHE_HOME
 +
* pacaur: $XDG_CACHE_HOME (changed from /tmp, see [https://github.com/rmarquis/pacaur/commit/c5d750f75f040b21249fff100a2c8875348d03d1])
 +
* bauerbill: $PWD/build
 +
* pkgbuilder: $PWD, /tmp when specified with -S
 +
* packer: /tmp (TMPDIR)
 +
* yaourt: /tmp (yaourtrc)
  
[[User:Tuxce|Tuxce]] ([[User talk:Tuxce|talk]]) 12:50, 26 April 2013 (UTC)
+
-- [[User:Alad|Alad]] ([[User talk:Alad|talk]]) 18:16, 1 April 2016 (UTC)
: I think it only means asking the user to look and check PKGBUILD, especially for download URL. So it can be renamed to "Check PKGBUILD". -- [[User:Fengchao|Fengchao]] ([[User talk:Fengchao|talk]]) 00:13, 27 April 2013 (UTC)
+
::My guess is that the "Secure" column is an adaptation of the "Manually Parses PKGBUILD*" column in [https://wiki.archlinux.org/index.php?title=AUR_Helpers&oldid=245047#Comparison_Table this old revision], see also the note at the bottom. -- [[User:Kynikos|Kynikos]] ([[User talk:Kynikos|talk]]) 06:50, 28 April 2013 (UTC)
+
:::Given that at the end, all AUR helpers (exept cower) call makepkg, PKGBUILD are sourced, so I think it should be removed. The word "secure" is just confusing.
+
:::For example, aurget can be considered more "secure" than owl or aura as it ask to review PKGBUILD before it being sourced.
+
:::[[User:Tuxce|Tuxce]] ([[User talk:Tuxce|talk]]) 20:05, 28 April 2013 (UTC)
+
::::Agreed, "Secure" without any kind of explanation doesn't mean anything. -- [[User:Kynikos|Kynikos]] ([[User talk:Kynikos|talk]]) 11:43, 29 April 2013 (UTC)
+
  
::::: "Secure" simply means the PKGBUILDs aren't sourced ***before*** the user has a chance to inspect the PKGBUILD himself. Makepkg does source the PKGBUILD obviously, it doesn't mean using it is insecure (but using it blindly is). For example, packer source the PKGBUILD before showing it to the user, unless the --preview option is passed. And so does pacaur (when using the bash solver), although the PKGBUILDs are scanned for potential malicious pseudo code using sudo. Spyhawk 12:07, 15 May 2013 (UTC)
+
: Yes, this could be useful. Although you'd want not to use color here, since users that know what they're doing would prefer to use /tmp (or setting up BUILDDIR to /tmp). --[[User:Spyhawk|Spyhawk]] ([[User talk:Spyhawk|talk]]) 11:15, 3 April 2016 (UTC)
  
::::::So, just to include also cower in the definition, I think a more correct formulation would be: ''"Secure means that the application, by default, doesn't source the PKGBUILD at all, or, before doing it, reminds the user and offers him the opportunity to inspect it manually"''.
+
:: +1. see also [[#Multi-thread support]]. --[[User:Indigo|Indigo]] ([[User talk:Indigo|talk]]) 11:33, 3 April 2016 (UTC)
::::::Note though that the inspection of a PKGBUILD is always a separate human operation that the user has to do deliberately, and it's independent of the helper being used; this means that every "secure" application can be used insecurely if the user doesn't inspect the PKGBUILD, and vice versa every "insecure" application can be used securely if e.g. the user inspects the PKGBUILD through the AUR website.
+
::: Well, while it does have benefits for some users, it's still a bad default. As you say though, this is easy enough to change either way, unlike any of the behaviour described in the other columns.
::::::Also, the "by default" clause is IMHO very important, in fact you could for example use packer with an alias that runs it with the --preview flag, thus making it a "secure" application, with just such a minimal change.
+
::: We could leave out the colors, but mention the drawbacks/benefits in the "meanings" paragraph. -- [[User:Alad|Alad]] ([[User talk:Alad|talk]]) 13:35, 4 April 2016 (UTC)
::::::By the way, I haven't used yaourt for a while, but IIRC it used to let the user review the PKGBUILD after downloading it; it's not clear why it's not considered secure.
+
::::::In the end, my opinion is that every application offers different degrees of security, and trying to sum all up in a Yes/No column is too simplistic: I would leave more verbose security considerations in the descriptions of every application above the table, or at least I would add some words in the "Specificity" column.
+
:::::: -- [[User:Kynikos|Kynikos]] ([[User talk:Kynikos|talk]]) 16:41, 18 May 2013 (UTC)
+
  
::::::: I like your definition, but a shorter one would be welcome (if that is possible?). Of course the security of helper heavily depends on the user, but it is expected to take his full responsibility and check the PKGBUILDs. An "insecure" helper simply has a security flaw, independently of the user.
+
:::: It is bad default because some users have no idea about what they are doing, but this is strictly related to user preferences. Adding the meaning instead of colors sounds like the perfect solution to me. --[[User:Spyhawk|Spyhawk]] ([[User talk:Spyhawk|talk]]) 14:35, 4 April 2016 (UTC).
::::::: Yaourt does scan dependencies before letting the user having a look at the PKGBUILDs, so that is similar to what packer does by default. However, yaourt seems to do some other step in between but I haven't been able to understand why and for which purpose (yaourt's code is a bit cryptic to me, Tuxce might better explain what this is fully about here).
+
::::::: I do agree that it is hard to summarize the security aspect with a "Yes/No" box only, and so is the accuracy of the dependencies solver. Security is always done at the expense of the efficiency of the helper, and actually the "fully secure" helper are also the worst in solving dependencies. On the other hand, bash solvers are fully accurate, but are the less secure. Hopefully, this issue will be solved soon and the JSON rpc interface will become much more reliable, so helper could entirely rely on it instead of looking at the downloaded PKGBUILDs.
+
::::::: [[User:Spyhawk|Spyhawk]] ([[User talk:Spyhawk|talk]]) 12:18, 19 May 2013 (UTC)
+
  
::::::::yaourt doesn't parse PKGBUILD before user can read them (and never did).
+
== Multi-thread support ==
::::::::[[User:Tuxce|Tuxce]] ([[User talk:Tuxce|talk]]) 13:28, 25 June 2013 (UTC)
+
  
::::::::The definition I've proposed is just my attempt to interpret, in a more coherent way, the idea of "secure" you're using in the table: I don't think it can be made shorter than that, also because IMHO it's too biased as it analyzes only a little part of the problem, thus oversimplifying it.
+
This also made me wonder if tools differentiate regarding multi-thread support (seems related, e.g. cower has a defaulted option for it). --[[User:Indigo|Indigo]] ([[User talk:Indigo|talk]]) 11:33, 3 April 2016 (UTC)
::::::::I still don't get what you exactly mean with ''An "insecure" helper simply has a security flaw, independently of the user'': what is this user-independent ''security flaw''? If it's just the fact that the application doesn't remind the user to check the PKGBUILD manually, personally I wouldn't consider it a security flaw, in fact I think the real security flaw, which is intrinsic to the AUR, is in the fact that the user, even before launching the helper, 1) must be aware that the helper is going to source some third-party code on his machine, and 2) must be able to understand that code in order to decide if it's safe or not; now, every user should be aware of this when installing AUR packages, and the fact that the helper reminds the user or not does really make a minimal difference in terms of security.
+
::::::::-- [[User:Kynikos|Kynikos]] ([[User talk:Kynikos|talk]]) 13:50, 20 May 2013 (UTC)
+
  
::::::::: I'm not referring to any situation where the user does not check the PKGBUILDs (or check but still continues despite the PKGBUILD being malicious). As previously written above, I'm referring to security flaw that sources the PKGBUILDs ***before*** the user is asked to view the PKGBUILDs himself. So a malicious code would be executed even before the user has a look at the PKGBUILD. Packer (without --preview) is the most stunning example here.
+
: AFAIK, besides cower, packer [http://kmkeen.com/multithreaded-bash/] and bauerbill ({{ic|download.sh}} amongst others) have multiple threads. aurutils also uses aria2c for downloads, if that counts.
::::::::: [[User:Spyhawk|Spyhawk]] ([[User talk:Spyhawk|talk]]) 18:28, 20 May 2013 (UTC)
+
: The benefits of multiple threads are however not always clear:  
 +
:: * by my understanding, cower uses multiple threads, but with one query per package [https://github.com/falconindy/cower/blob/master/cower.c#L667] (compare against multiinfo).
 +
:: * More generally, tasks (like dependency solving) can be sped up by using different methods which need to be called less often
 +
:: * Building packages would almost always be done sequentially: dependencies have to be installed (resulting in pacman locks), and there's {{ic|-j}} in {{ic|makepkg.conf}} anyway.
 +
: Regardless, there are some large differences in AUR helper speed (with bauerbill being ahead of the rest). But I'm not sure how to quantify this in the table ... -- [[User:Alad|Alad]] ([[User talk:Alad|talk]]) 12:31, 3 April 2016 (UTC)
  
::::::::::Well, it seems we've fallen into a loop we can't really break: each of us has exposed enough arguments, let's leave this discussion open for a while and see if somebody else supports one or the other side.
+
:: Multi-thread support doesn't necessarily mean the helper is better. In cower case, multi-thread support was implemented before multiinfo was available in the RPC interface, and as of today using multiinfo is less complex and faster than using multiple info threads. Since it is difficult to implement multiinfo support without an important rewrite, cower multithreading is more a drawback than an advantage.
::::::::::Meanwhile I've reworded the note, I think it's clearer now.
+
:: As for speed, it's indeed very hard to quantify in a meaningful manner. For example, pacaur dependency solver is slower than bauerbill's solver, but on the other hand it is designed to compute more stuff than other helpers up front in order to avoid bothering the user once the install process is started. --[[User:Spyhawk|Spyhawk]] ([[User talk:Spyhawk|talk]]) 13:42, 3 April 2016 (UTC)
::::::::::-- [[User:Kynikos|Kynikos]] ([[User talk:Kynikos|talk]]) 14:14, 22 May 2013 (UTC)
+
::::::::::: Does the secure definition agreed among the developer and user base? Base on the discussion, the answer is no.
+
::::::::::: I myself do not think the time of sourcing PKGBUILD can be the seperate line between secure and no secure. If source PKGBUILD is danger, then source the PKGBUILD before user checking is 100% danger and after is 95% danger. Most people do not check the PKGBUILD, they just install blindly. Even if they do, few of the could find the danger hiden there.
+
::::::::::: So my suggestion is remove secure column until the definition is well understood between developer and user community.
+
::::::::::: -- [[User:Fengchao|Fengchao]] ([[User talk:Fengchao|talk]]) 23:09, 30 May 2013 (UTC)
+
  
:::::::::::: The fact that some people don't check the PKGBUILD is irrelevant, since an Arch user is given complete responsibility over its system. In contrast, some helpers have clearly a dangerous security flaw.
+
::: Interesting. Actually, I did not want to induce a "speed" column, rather the opposite. As you both say, always very difficult to choose a fairly universal/comparable benchmark, so "speed" as such is better be left out of comparison (as a column). If one wants to mention it, it might be useful to have a general remark at the top of the table, or somewhere else in the article, quoting some of the influencing factors you name; perhaps linking to (re -j) [[Makepkg#MAKEFLAGS]] and (re Skyhawk's remark above) [[Makepkg#Improving compile times]]. --[[User:Indigo|Indigo]] ([[User talk:Indigo|talk]]) 14:01, 3 April 2016 (UTC)
:::::::::::: [[User:Spyhawk|Spyhawk]] ([[User talk:Spyhawk|talk]]) 21:51, 1 June 2013 (UTC)
+
 
 +
== <s>Comparison table - solver: new test-case</s> ==
 +
 
 +
{{AUR|ros-indigo-desktop}} has 201 AUR dependencies. Compared to {{AUR|plasma-meta-git}}, this shows another limitation of the AUR: RPC queries are done via GET requests, and querying many packages may result in an HTTP 414 error. {{AUR|pacaur}} ([https://github.com/rmarquis/pacaur/blob/9c5d3039dcda9028d39a4c85281f14afc27330c9/pacaur#L1131]), {{AUR|bauerbill}} ([http://xyne.archlinux.ca/projects/python3-aur/], 2016-02-01) and {{AUR|aurutils}} ([https://github.com/AladW/aurutils/blob/master/bin/aursearch#L63]) workaround this by splitting info queries.
 +
 
 +
That said, you could presumably avoid this issue altogether by not querying duplicates.
 +
 
 +
Results:
 +
 
 +
* {{AUR|aurutils}}: fails [https://github.com/AladW/aurutils/issues/111]
 +
* {{AUR|pkgbuilder}}: works
 +
* {{AUR|pacaur}}: works (build not tested)
 +
* {{AUR|bauerbill}}: not tested
 +
 
 +
I didn't bother with the others as they already failed on {{AUR|plasma-meta-git}}. -- [[User:Alad|Alad]] ([[User talk:Alad|talk]]) 10:57, 21 April 2016 (UTC)
 +
 
 +
:{{AUR|aurutils}} now works as well. [https://github.com/AladW/aurutils/commit/7749b8329ec0a7898323e8d8ddd97072a5152c35] [https://github.com/AladW/aurutils/commit/90acbdb56f5ec6d40264f3cfad8a4f029d95301a] {{AUR|bauerbill}} seems to miss some dependencies, but a forum report will probably fix this. As such, the new test-case wouldn't change the table results, so I'm going to close this one.

Latest revision as of 11:58, 17 May 2016

Note: Moderation — If your AUR helper does partial upgrades without explicit user intervention (i.e, specifying -Sy on the command line), it has no place on this page or anywhere else on ArchWiki. No exceptions. -- Alad (talk) 09:37, 20 September 2015 (UTC)

Comparison table - build directory

Considering /tmp is mounted as tmpfs on Arch, and the potential downsides from building in RAM (running out of space), I think a column with the default build location for various helpers would be helpful.

The default values I've garnered so far, assuming TMPDIR is not set:

  • aurutils: $XDG_CACHE_HOME
  • pacaur: $XDG_CACHE_HOME (changed from /tmp, see [1])
  • bauerbill: $PWD/build
  • pkgbuilder: $PWD, /tmp when specified with -S
  • packer: /tmp (TMPDIR)
  • yaourt: /tmp (yaourtrc)

-- Alad (talk) 18:16, 1 April 2016 (UTC)

Yes, this could be useful. Although you'd want not to use color here, since users that know what they're doing would prefer to use /tmp (or setting up BUILDDIR to /tmp). --Spyhawk (talk) 11:15, 3 April 2016 (UTC)
+1. see also #Multi-thread support. --Indigo (talk) 11:33, 3 April 2016 (UTC)
Well, while it does have benefits for some users, it's still a bad default. As you say though, this is easy enough to change either way, unlike any of the behaviour described in the other columns.
We could leave out the colors, but mention the drawbacks/benefits in the "meanings" paragraph. -- Alad (talk) 13:35, 4 April 2016 (UTC)
It is bad default because some users have no idea about what they are doing, but this is strictly related to user preferences. Adding the meaning instead of colors sounds like the perfect solution to me. --Spyhawk (talk) 14:35, 4 April 2016 (UTC).

Multi-thread support

This also made me wonder if tools differentiate regarding multi-thread support (seems related, e.g. cower has a defaulted option for it). --Indigo (talk) 11:33, 3 April 2016 (UTC)

AFAIK, besides cower, packer [2] and bauerbill (download.sh amongst others) have multiple threads. aurutils also uses aria2c for downloads, if that counts.
The benefits of multiple threads are however not always clear:
* by my understanding, cower uses multiple threads, but with one query per package [3] (compare against multiinfo).
* More generally, tasks (like dependency solving) can be sped up by using different methods which need to be called less often
* Building packages would almost always be done sequentially: dependencies have to be installed (resulting in pacman locks), and there's -j in makepkg.conf anyway.
Regardless, there are some large differences in AUR helper speed (with bauerbill being ahead of the rest). But I'm not sure how to quantify this in the table ... -- Alad (talk) 12:31, 3 April 2016 (UTC)
Multi-thread support doesn't necessarily mean the helper is better. In cower case, multi-thread support was implemented before multiinfo was available in the RPC interface, and as of today using multiinfo is less complex and faster than using multiple info threads. Since it is difficult to implement multiinfo support without an important rewrite, cower multithreading is more a drawback than an advantage.
As for speed, it's indeed very hard to quantify in a meaningful manner. For example, pacaur dependency solver is slower than bauerbill's solver, but on the other hand it is designed to compute more stuff than other helpers up front in order to avoid bothering the user once the install process is started. --Spyhawk (talk) 13:42, 3 April 2016 (UTC)
Interesting. Actually, I did not want to induce a "speed" column, rather the opposite. As you both say, always very difficult to choose a fairly universal/comparable benchmark, so "speed" as such is better be left out of comparison (as a column). If one wants to mention it, it might be useful to have a general remark at the top of the table, or somewhere else in the article, quoting some of the influencing factors you name; perhaps linking to (re -j) Makepkg#MAKEFLAGS and (re Skyhawk's remark above) Makepkg#Improving compile times. --Indigo (talk) 14:01, 3 April 2016 (UTC)

Comparison table - solver: new test-case

ros-indigo-desktopAUR has 201 AUR dependencies. Compared to plasma-meta-gitAUR, this shows another limitation of the AUR: RPC queries are done via GET requests, and querying many packages may result in an HTTP 414 error. pacaurAUR ([4]), bauerbillAUR ([5], 2016-02-01) and aurutilsAUR ([6]) workaround this by splitting info queries.

That said, you could presumably avoid this issue altogether by not querying duplicates.

Results:

I didn't bother with the others as they already failed on plasma-meta-gitAUR. -- Alad (talk) 10:57, 21 April 2016 (UTC)

aurutilsAUR now works as well. [8] [9] bauerbillAUR seems to miss some dependencies, but a forum report will probably fix this. As such, the new test-case wouldn't change the table results, so I'm going to close this one.