Difference between revisions of "Talk:AUR helpers"

From ArchWiki
Jump to: navigation, search
(Comparison table - solver: new test-case: remove details)
 
(172 intermediate revisions by 10 users not shown)
Line 1: Line 1:
Authors of each front end should post a short (2-3 line) description of their creation, along with a homepage link and an AUR link (where applicable). A link to a screenshot page would also be nice (if applicable).
+
{{Note|'''Moderation''' — If your AUR helper does [[partial upgrade]]s ''without explicit user intervention'' (i.e, specifying {{ic|-Sy}} on the command line), it has no place on this page or anywhere else on ArchWiki. No exceptions. -- [[User:Alad|Alad]] ([[User talk:Alad|talk]]) 09:37, 20 September 2015 (UTC)
 +
}}
  
== Comparison table removal ==
+
== Comparison table - build directory ==
I see the comparison table has been removed, without any comment. The question I'm asking myself currently is "Do we really need to have it back?" The page looks fine as it is now. Spyhawk 13:03, 7 February 2013 (UTC)
+
  
== More relevant AUR comparison table ==
+
Considering /tmp is mounted as tmpfs on Arch, and the potential downsides from building in RAM (running out of space), I think a column with the default build location for various helpers would be helpful.
  
Here is a proposal for a shorter, and more relevant AUR comparison table.
+
The default values I've garnered so far, assuming TMPDIR is not set:
  
* Removal of all columns featuring AUR features that are expected from any AUR helper (Handles Deps, AUR search, Handles Upgrades). Those are common to all AUR helper anyway.
+
* aurutils: $XDG_CACHE_HOME
* Added a Yes/No column for "Usage similar to pacman". This is indicative, and I believe the various "see man helper" are useless (a specific column could be added if necessary).
+
* pacaur: $XDG_CACHE_HOME (changed from /tmp, see [https://github.com/rmarquis/pacaur/commit/c5d750f75f040b21249fff100a2c8875348d03d1])
* Replaced the column "Usage" by a more descriptive "Specificity" column. Use it to highlight specific focus or strength of the helper.
+
* bauerbill: $PWD/build
 +
* pkgbuilder: $PWD, /tmp when specified with -S
 +
* packer: /tmp (TMPDIR)
 +
* yaourt: /tmp (yaourtrc)
  
=== Comparison Table ===
+
-- [[User:Alad|Alad]] ([[User talk:Alad|talk]]) 18:16, 1 April 2016 (UTC)
  
{| border="1" cellpadding="4" cellspacing="0"
+
: Yes, this could be useful. Although you'd want not to use color here, since users that know what they're doing would prefer to use /tmp (or setting up BUILDDIR to /tmp). --[[User:Spyhawk|Spyhawk]] ([[User talk:Spyhawk|talk]]) 11:15, 3 April 2016 (UTC)
! Name !! Written in !! Official Repo support !! Usage similar to pacman !! Shell Tab Completion !! Manually Parses PKGBUILD* || Multilingual !! Active Project !! Specificity
+
|-
+
! [[aura]]
+
| Haskell || {{Yes}} || {{No}} || Bash/zsh || {{Yes}} || {{Yes}} || {{Yes}} || Handle Backups, Downgrades
+
|-
+
! aurget
+
| Bash || {{No}} || {{Yes}} || Bash || {{No}} || {{No}} || {{Yes}} || -
+
|-
+
! aurora
+
| Python3 || {{No}} || {{No}} || {{No}} || {{No}}  || {{No}} || {{Yes}} || -
+
|-
+
! cower
+
| C|| {{No}} || {{No}} || Bash/zsh || {{Yes}} || {{No}} || {{Yes}} || Minimalist helper without automatic build support.
+
|-
+
! owl
+
| Dash || {{Yes}} || {{No}} || Bash || {{Yes}} || {{No}} || {{Yes}} || -
+
|-
+
! [[pacaur]]
+
| Bash/C || {{Yes}} || {{Yes}} || Bash || optional|| {{No}} || {{Yes}} || Minimize user interaction.
+
|-
+
! packer
+
| Bash || {{Yes}} || {{Yes}} || {{No}} || {{No}} || {{No}} || {{Yes}} || -
+
|-
+
! paktahn
+
| Lisp|| {{Yes}} || {{Yes}} || {{No}} || {{No}} || {{No}} || {{Yes}} || -
+
|-
+
! pbfetch
+
| Bash || {{Yes}} || {{Yes}} || {{No}} || {{No}} || {{No}} || {{Yes}} || -
+
|-
+
! PKGBUILDer
+
| Python3 || {{Yes}} ({{Ic|pb}} command) || {{No}} || {{No}} || {{No}} || {{Yes}} || {{Yes}} || -
+
|-
+
! spinach
+
| Bash || {{Yes}} || {{No}} || {{No}} || {{Yes}} || {{No}} || {{Yes}} || -
+
|-
+
! [[yaourt]]
+
| Bash/C || {{Yes}} || {{Yes}} || Bash/zsh/fish || {{No}} || {{Yes}} || {{Yes}} || Handle Backups, ABS support
+
|}
+
  
{{Note|Scripts that do not parse PKGBUILDs manually opt instead to execute them directly for their variables. This is not considered secure, but is generally more accurate than manual parsing.}}
+
:: +1. see also [[#Multi-thread support]]. --[[User:Indigo|Indigo]] ([[User talk:Indigo|talk]]) 11:33, 3 April 2016 (UTC)
 +
::: Well, while it does have benefits for some users, it's still a bad default. As you say though, this is easy enough to change either way, unlike any of the behaviour described in the other columns.
 +
::: We could leave out the colors, but mention the drawbacks/benefits in the "meanings" paragraph. -- [[User:Alad|Alad]] ([[User talk:Alad|talk]]) 13:35, 4 April 2016 (UTC)
  
Any other useful ideas? Spyhawk
+
:::: It is bad default because some users have no idea about what they are doing, but this is strictly related to user preferences. Adding the meaning instead of colors sounds like the perfect solution to me. --[[User:Spyhawk|Spyhawk]] ([[User talk:Spyhawk|talk]]) 14:35, 4 April 2016 (UTC).
  
:Removing redundant fields: Yes.
+
== Multi-thread support ==
:Usage similar to pacman column: Yes. I'd argue aura is similar to pacman, though. Compared to say, cower or owl, it is.
+
:Usage column: Yes, this was totally useless.
+
:This revision is less cluttered, and thus easier to understand. I like it!
+
  
::2./ is actually a hard one. It depends on what is your definition of "similar". I'd say that helpers that install with a "-S" command are similar, but this is debatable. I guess this should be debated on the wiki Talk page.
+
This also made me wonder if tools differentiate regarding multi-thread support (seems related, e.g. cower has a defaulted option for it). --[[User:Indigo|Indigo]] ([[User talk:Indigo|talk]]) 11:33, 3 April 2016 (UTC)
::Another point I'd like to improve is the "manually parsed" column through a more general "secure" column... but how do you define that an helper is secure or not?
+
::Do you know any other people that would be interested in helping to improve this table? Spyhawk
+
  
::: I'd say:
+
: AFAIK, besides cower, packer [http://kmkeen.com/multithreaded-bash/] and bauerbill ({{ic|download.sh}} amongst others) have multiple threads. aurutils also uses aria2c for downloads, if that counts.
:::helper --install package # Not similar to pacman.
+
: The benefits of multiple threads are however not always clear:  
:::helper -S package  # Obviously the same as pacman.
+
:: * by my understanding, cower uses multiple threads, but with one query per package [https://github.com/falconindy/cower/blob/master/cower.c#L667] (compare against multiinfo).
:::helper -A package  # Similar enough, consider it has `-i` and `-s` too.
+
:: * More generally, tasks (like dependency solving) can be sped up by using different methods which need to be called less often
:::We could say a helper is secure if it tries to protect you from rogue PKGBUILDs. I saw one (can't remember which) that actually scanned for instances of the word sudo. I think it still sourced the PKGBUILD, but at least it tried. Obviously manually parsing the PKGBUILD (and the .install file?) would be the most secure.
+
:: * Building packages would almost always be done sequentially: dependencies have to be installed (resulting in pacman locks), and there's {{ic|-j}} in {{ic|makepkg.conf}} anyway.
:::As for people to help us... would the yaourt people care? I could see some of the upstarts (spinach, owl?) being cooperative. cower seems to be a powerhouse, so we should talk to them as well. packer has infrequent updates (looking at it's github network) and I don't know if they'd get back to us.
+
: Regardless, there are some large differences in AUR helper speed (with bauerbill being ahead of the rest). But I'm not sure how to quantify this in the table ... -- [[User:Alad|Alad]] ([[User talk:Alad|talk]]) 12:31, 3 April 2016 (UTC)
  
:::: The aur helper that scans the PKGBUILD is spinach, but it does manual parsing anyway. I've also seen that security feature, and implemented a very similar one in pacaur. Packer does source the PKGBUILD before asking for editing, so that's clearly insecure (unless using --preview, but that is not set by default). Pbfetch removes everything after build() before sourcing. I guess that would be considered secure too. I can't tell for yaourt, the code is too cryptic to me. Spyhawk
+
:: Multi-thread support doesn't necessarily mean the helper is better. In cower case, multi-thread support was implemented before multiinfo was available in the RPC interface, and as of today using multiinfo is less complex and faster than using multiple info threads. Since it is difficult to implement multiinfo support without an important rewrite, cower multithreading is more a drawback than an advantage.
 +
:: As for speed, it's indeed very hard to quantify in a meaningful manner. For example, pacaur dependency solver is slower than bauerbill's solver, but on the other hand it is designed to compute more stuff than other helpers up front in order to avoid bothering the user once the install process is started. --[[User:Spyhawk|Spyhawk]] ([[User talk:Spyhawk|talk]]) 13:42, 3 April 2016 (UTC)
  
Here is an improved, but incomplete table:
+
::: Interesting. Actually, I did not want to induce a "speed" column, rather the opposite. As you both say, always very difficult to choose a fairly universal/comparable benchmark, so "speed" as such is better be left out of comparison (as a column). If one wants to mention it, it might be useful to have a general remark at the top of the table, or somewhere else in the article, quoting some of the influencing factors you name; perhaps linking to (re -j) [[Makepkg#MAKEFLAGS]] and (re Skyhawk's remark above) [[Makepkg#Improving compile times]]. --[[User:Indigo|Indigo]] ([[User talk:Indigo|talk]]) 14:01, 3 April 2016 (UTC)
  
{{note|'''Secure''' means that the AUR helper tries to protect from malicious PKGBUILD, using manual parsing or other means.}}
+
== <s>Comparison table - solver: new test-case</s> ==
  
{| border="1" cellpadding="4" cellspacing="0"
+
{{AUR|ros-indigo-desktop}} has 201 AUR dependencies. Compared to {{AUR|plasma-meta-git}}, this shows another limitation of the AUR: RPC queries are done via GET requests, and querying many packages may result in an HTTP 414 error. {{AUR|pacaur}} ([https://github.com/rmarquis/pacaur/blob/9c5d3039dcda9028d39a4c85281f14afc27330c9/pacaur#L1131]), {{AUR|bauerbill}} ([http://xyne.archlinux.ca/projects/python3-aur/], 2016-02-01) and {{AUR|aurutils}} ([https://github.com/AladW/aurutils/blob/master/bin/aursearch#L63]) workaround this by splitting info queries.
! Name !! Written in !! Active Project !! Official Repo support !! Syntax similar to pacman !! Shell Tab Completion !! Secure !! Multilingual !! Specificity
+
|-
+
! [[aura]]
+
| Haskell || {{Yes}} || {{Yes}} || {{Yes}} || Bash/zsh || {{Yes}} || {{Yes}}  || Handle Backups, Downgrades
+
|-
+
! aurget
+
| Bash || {{Yes}} || {{No}} || {{Yes}} || Bash || ? || {{No}} || -
+
|-
+
! aurora
+
| Python3 || {{Yes}} || {{No}} || {{No}} || {{No}} || ?  || {{No}} || -
+
|-
+
! cower
+
| C || {{Yes}} || {{No}} || {{No}} || Bash/zsh || {{Yes}} || {{No}} || Minimalist helper without automatic build support.
+
|-
+
! owl
+
| Dash || {{Yes}} || {{Yes}} || {{No}} || Bash || {{Yes}} || {{No}} || -
+
|-
+
! [[pacaur]]
+
| Bash/C || {{Yes}} || {{Yes}} || {{Yes}} || Bash || {{Yes}} || {{No}} || Minimize user interaction.
+
|-
+
! packer
+
| Bash || {{Yes}} || {{Yes}} || {{Yes}} || {{No}} || {{No}} || {{No}} || -
+
|-
+
! paktahn
+
| Lisp || {{Yes}} || {{Yes}} || {{Yes}} || {{No}} || ? || {{No}} || -
+
|-
+
! pbfetch
+
| Bash || {{Yes}} || {{Yes}} || {{Yes}} || {{No}} || {{Yes}} || {{No}} || -
+
|-
+
! PKGBUILDer
+
| Python3 || {{Yes}} || {{Yes}} ({{Ic|pb}} command) || {{No}} || {{No}} || ? || {{Yes}} || -
+
|-
+
! spinach
+
| Bash || {{Yes}} || {{Yes}} || {{No}} || {{No}} || {{Yes}} || {{No}} || -
+
|-
+
! [[yaourt]]
+
| Bash/C || {{Yes}} || {{Yes}} || {{Yes}} || Bash/zsh/fish || {{No}} || {{Yes}} || Handle Backups, ABS support
+
|}
+
  
It's important that people looking at this know what "Secure" means, so I like that the note is at the top. Do you think a numbered rating system would work better than just Yes / No?
+
That said, you could presumably avoid this issue altogether by not querying duplicates.
Also, the last time I checked, yaourt just shamelessly sources the PKGBUILD. -fosskers
+
 
: Maybe a rating system would be good, if objective. What would be the criteria to look at? Spyhawk 11:56, 5 February 2013 (UTC)
+
Results:
 +
 
 +
* {{AUR|aurutils}}: fails [https://github.com/AladW/aurutils/issues/111]
 +
* {{AUR|pkgbuilder}}: works
 +
* {{AUR|pacaur}}: works (build not tested)
 +
* {{AUR|bauerbill}}: not tested
 +
 
 +
I didn't bother with the others as they already failed on {{AUR|plasma-meta-git}}. -- [[User:Alad|Alad]] ([[User talk:Alad|talk]]) 10:57, 21 April 2016 (UTC)
 +
 
 +
:{{AUR|aurutils}} now works as well. [https://github.com/AladW/aurutils/commit/7749b8329ec0a7898323e8d8ddd97072a5152c35] [https://github.com/AladW/aurutils/commit/90acbdb56f5ec6d40264f3cfad8a4f029d95301a] {{AUR|bauerbill}} seems to miss some dependencies, but a forum report will probably fix this. As such, the new test-case wouldn't change the table results, so I'm going to close this one.

Latest revision as of 11:58, 17 May 2016

Note: Moderation — If your AUR helper does partial upgrades without explicit user intervention (i.e, specifying -Sy on the command line), it has no place on this page or anywhere else on ArchWiki. No exceptions. -- Alad (talk) 09:37, 20 September 2015 (UTC)

Comparison table - build directory

Considering /tmp is mounted as tmpfs on Arch, and the potential downsides from building in RAM (running out of space), I think a column with the default build location for various helpers would be helpful.

The default values I've garnered so far, assuming TMPDIR is not set:

  • aurutils: $XDG_CACHE_HOME
  • pacaur: $XDG_CACHE_HOME (changed from /tmp, see [1])
  • bauerbill: $PWD/build
  • pkgbuilder: $PWD, /tmp when specified with -S
  • packer: /tmp (TMPDIR)
  • yaourt: /tmp (yaourtrc)

-- Alad (talk) 18:16, 1 April 2016 (UTC)

Yes, this could be useful. Although you'd want not to use color here, since users that know what they're doing would prefer to use /tmp (or setting up BUILDDIR to /tmp). --Spyhawk (talk) 11:15, 3 April 2016 (UTC)
+1. see also #Multi-thread support. --Indigo (talk) 11:33, 3 April 2016 (UTC)
Well, while it does have benefits for some users, it's still a bad default. As you say though, this is easy enough to change either way, unlike any of the behaviour described in the other columns.
We could leave out the colors, but mention the drawbacks/benefits in the "meanings" paragraph. -- Alad (talk) 13:35, 4 April 2016 (UTC)
It is bad default because some users have no idea about what they are doing, but this is strictly related to user preferences. Adding the meaning instead of colors sounds like the perfect solution to me. --Spyhawk (talk) 14:35, 4 April 2016 (UTC).

Multi-thread support

This also made me wonder if tools differentiate regarding multi-thread support (seems related, e.g. cower has a defaulted option for it). --Indigo (talk) 11:33, 3 April 2016 (UTC)

AFAIK, besides cower, packer [2] and bauerbill (download.sh amongst others) have multiple threads. aurutils also uses aria2c for downloads, if that counts.
The benefits of multiple threads are however not always clear:
* by my understanding, cower uses multiple threads, but with one query per package [3] (compare against multiinfo).
* More generally, tasks (like dependency solving) can be sped up by using different methods which need to be called less often
* Building packages would almost always be done sequentially: dependencies have to be installed (resulting in pacman locks), and there's -j in makepkg.conf anyway.
Regardless, there are some large differences in AUR helper speed (with bauerbill being ahead of the rest). But I'm not sure how to quantify this in the table ... -- Alad (talk) 12:31, 3 April 2016 (UTC)
Multi-thread support doesn't necessarily mean the helper is better. In cower case, multi-thread support was implemented before multiinfo was available in the RPC interface, and as of today using multiinfo is less complex and faster than using multiple info threads. Since it is difficult to implement multiinfo support without an important rewrite, cower multithreading is more a drawback than an advantage.
As for speed, it's indeed very hard to quantify in a meaningful manner. For example, pacaur dependency solver is slower than bauerbill's solver, but on the other hand it is designed to compute more stuff than other helpers up front in order to avoid bothering the user once the install process is started. --Spyhawk (talk) 13:42, 3 April 2016 (UTC)
Interesting. Actually, I did not want to induce a "speed" column, rather the opposite. As you both say, always very difficult to choose a fairly universal/comparable benchmark, so "speed" as such is better be left out of comparison (as a column). If one wants to mention it, it might be useful to have a general remark at the top of the table, or somewhere else in the article, quoting some of the influencing factors you name; perhaps linking to (re -j) Makepkg#MAKEFLAGS and (re Skyhawk's remark above) Makepkg#Improving compile times. --Indigo (talk) 14:01, 3 April 2016 (UTC)

Comparison table - solver: new test-case

ros-indigo-desktopAUR has 201 AUR dependencies. Compared to plasma-meta-gitAUR, this shows another limitation of the AUR: RPC queries are done via GET requests, and querying many packages may result in an HTTP 414 error. pacaurAUR ([4]), bauerbillAUR ([5], 2016-02-01) and aurutilsAUR ([6]) workaround this by splitting info queries.

That said, you could presumably avoid this issue altogether by not querying duplicates.

Results:

I didn't bother with the others as they already failed on plasma-meta-gitAUR. -- Alad (talk) 10:57, 21 April 2016 (UTC)

aurutilsAUR now works as well. [8] [9] bauerbillAUR seems to miss some dependencies, but a forum report will probably fix this. As such, the new test-case wouldn't change the table results, so I'm going to close this one.