Improve pacman performance

From ArchWiki
Revision as of 22:31, 31 January 2007 by Xabbott (talk | contribs) (merged entry:Faster Pacman Downloads into this article)
Jump to navigation Jump to search


Pacman can be further optimized in two ways.

Improving database access speeds

Pacman stores all package information in a collection of small files, one for each package. Improving database access speeds reduces the time taken in database-related tasks, e.g. searching packages and resolving package dependencies.

A first step is to run

pacman-optimize

as root. This will attempt to put all the small files together in one (physical) location on the hard disk so that the hard disk head does not have to move so much when accessing all the packages. This is a safe step.

ody has posted a script on the forum that replaces the current Pacman database with a loopback filesystem which ensures that all the small files continue to stay together on the hard disk. Several users have reported great improvements, but problems have also been reported so do not do this unless you are an expert user.

To use ody's script you must have a kernel compiled with loopback filesystem support. The default kernels already have this, so you only need to be concerned with this if you compile your own custom kernel.

Improving download speeds

Pacman's speed in downloading packages can be improved by using a different application to download packages instead of Pacman's built-in file downloaded.

In all cases, make sure you have the latest Pacman before doing any modifications.

pacman -Syu

Using wget

This is also very handy if you need more powerful proxy settings than pacman's built-in capabilities.

To use wget, first install it with pacman -S wget and then modify /etc/pacman.conf by adding the following line to the [options] section:

XferCommand = /usr/bin/wget -c --passive-ftp -c %u

Instead of putting wget parameters in /etc/pacman.conf, you can also modify the wget configuration file directly (the system-wide file is /etc/wgetrc, per user files are $HOME/.wgetrc).


Using aria2

According to the aria2 website, aria2 is "a download utility with resuming and segmented downloading. Supports HTTP/HTTPS/FTP/BitTorrent/Metalink." This means that you can make several HTTP/FTP connections to an Arch mirror at the same time, which should result in increased download speeds.

Install it with pacman -S aria2 and then edit /etc/pacman.conf by adding the following line to the [option] section:

XferCommand = /usr/bin/aria2c -s 2 -m 2 -o %o %u

Let's run over the options here:

  • /usr/bin/aria2c - the location of the aria2 application
  • -s 2 - use 2 concurrent connections (you can set this higher if you want, but it's not going to do a whole lot)
  • -m 2 - make 2 attempts to download the package per mirror
  • -o %o - output to the file pacman specifies
  • %u - download the file pacman specifies


Aria2 Mirror Script

This script will greatly improve the download speed for broadband users. It takes the download url from pacman, then looks up the mirror list in /etc/pacman.d/<repo> and adds them all as mirrors to aria2. What ends up happening is aria2 connects to 10-20 servers downloading from all of them at the same time, which should give anyone on broadband a huge boost in download speed. This should max out the download speed for most people.

 #!/bin/bash
 #pacget script
 
 rm -f *.aria2
 cachedir=/var/cache/pacman/pkg
 file=`echo $1|awk -F '/' '{ print $NF }'`
 echo Downloading: $file|sed 's,/,,'
 repo=`echo $1|awk -F '/' '{x=NF-3 } { print  $x }'`
 echo repo: $repo 
 if [[ $2 = *.db.tar.gz.part || $repo = "testing" ]]; then
        echo "(using one mirror)"
        mirrors=$1
 else
        if [ -f /etc/pacman.d/$repo ]; then
        mirrors=`sed -n 's/^Server = //; s,$,/'"$file"', ; /^ftp/p; /^http/p' /etc/pacman.d/$repo`
        fi
 fi
 
 aria2c -t20 -m2 -l /var/log/pacget.log $mirrors -o $file.pacget
 if [ -f $cachedir/$file.pacget.aria2 ]; then
       rm $cachedir/$file.pacget $cachedir/$file.pacget.aria2
       exit 0
 fi
 
 mv $file.pacget $2
 exit 0

Save this script as /usr/bin/pacget.

chmod 755 /usr/bin/pacget

This makes the script an executable

In /etc/pacman.conf the following needs to be added:

XferCommand = /usr/bin/pacget %u %o

PS: If the script doesn't work try to make sure that the official archlinux.org mirror is the first one in your /etc/pacman.d/<repo> list. ie. /etc/pacman.d/

[current]
# North America
Server = ftp://ftp.archlinux.org/current/os/i686
...

Using other applications

There are other downloading applications that you can use with Pacman. Here they are, and their associated XferCommand settings:

  • snarf: XferCommand = /usr/bin/snarf -N %u
  • lftp: XferCommand = /usr/bin/lftp -c pget %u