Difference between revisions of "Improve pacman performance (简体中文)"

From ArchWiki
Jump to navigation Jump to search
Line 19: Line 19:
  
 
=== 进一步的优化 ===
 
=== 进一步的优化 ===
'''ody''' 曾经发表了 [http://bbs.archlinux.org/viewtopic.php?t=20385 a script on the forum] that replaces the current Pacman database with a ''loopback filesystem'' which ensures that all the small files continue to stay together on the hard disk. Several users have reported great improvements, but problems have also been reported so do not do this unless you are an expert user.
+
'''ody''' 曾经发表了 [http://bbs.archlinux.org/viewtopic.php?t=20385 a script on the forum] 这篇文章,将目前的Pacman数据库替换成了一个''回送文件系统(loopback filesystem)'',用来保证所有的小文件连续地存放在硬盘上。不少用户报告说会有很大的性能提升,但是也有问题出现。所以除非你是专业人士,否则不要这样做。
  
To use '''ody''''s script you must have a kernel compiled with loopback filesystem support. The default kernels already have this, so you only need to be concerned with this if you compile your own custom kernel.
+
要使用'''ody'''的脚本,你必须确定内核添加了"loopback filesystem"支持(内核一般默认支持)。
  
= Improving download speeds =
+
= 加快下载速度 =
  
Firstly, if your download speeds have been reduced to a crawl, ensure you are using one of the many [[mirrors]] and not ftp.archlinux.org, which, [http://www.archlinux.org/news/302/ as of March 2007], is now throttled.
+
如果你的包下载速度变得极慢,首先确定你用的是那些镜像([[mirrors]])网站而不是ftp.archlinux.org,因为后者根据[http://www.archlinux.org/news/302/ as of March 2007]所述,被限速了。
  
Pacman's speed in downloading packages can be improved by using a different application to download packages instead of Pacman's built-in file downloaded.
+
可以通过各种下载工具而不是Pacman内置的下载方式,来改善Pacman的下载速度。
  
In all cases, make sure you have the latest Pacman before doing any modifications.
+
不论怎样,在做任何修改前,你必须确定拥有了最新版的Pacman:
  
 
  pacman -Sy pacman
 
  pacman -Sy pacman
  
=== Using wget ===
+
=== 使用wget ===
  
This is also very handy if you need more powerful proxy settings than pacman's built-in capabilities.
+
对于需要更强大代理支持的用户来说,用wget比用Pacman自己的下载方式更加方便。
  
To use <code>wget</code>, first install it with <code>pacman -S wget</code> and then modify <code>/etc/pacman.conf</code> by adding the following line to the <code>[options]</code> section:
+
要使用 <code>wget</code>,首先使用<code>pacman -S wget</code>安装它,然后修改<code>/etc/pacman.conf</code>并在其中的<code>[options]</code>区段添加如下几行:
  
 
  XferCommand = /usr/bin/wget -c --passive-ftp -c %u
 
  XferCommand = /usr/bin/wget -c --passive-ftp -c %u
  
Instead of putting <code>wget</code> parameters in <code>/etc/pacman.conf</code>, you can also modify the <code>wget</code> configuration file directly (the system-wide file is <code>/etc/wgetrc</code>, per user files are <code>$HOME/.wgetrc</code>).
+
除了将<code>wget</code>参数放在<code>/etc/pacman.conf</code>里,你也可以直接修改<code>wget</code>配置文件(全局文件是<code>/etc/wgetrc</code>,各个用户的文件是<code>$HOME/.wgetrc</code>)
  
 
+
=== 使用aria2 ===
=== Using aria2 ===
 
  
 
According to the aria2 website, aria2 is "a download utility with resuming and segmented downloading. Supports HTTP/HTTPS/FTP/BitTorrent/Metalink." This means that you can make several HTTP/FTP connections to an Arch mirror at the same time, which should result in an increase in download speeds.
 
According to the aria2 website, aria2 is "a download utility with resuming and segmented downloading. Supports HTTP/HTTPS/FTP/BitTorrent/Metalink." This means that you can make several HTTP/FTP connections to an Arch mirror at the same time, which should result in an increase in download speeds.

Revision as of 13:00, 25 February 2008


Template:I18n links start Template:I18n entry Template:I18n entry Template:I18n links end


提高数据库连接速度

Pacman将所有软件包的信息放在一一对应的许多小文件中。通过改善数据库连接速度,可以减少花在数据库相关任务上的时间,比如:寻找软件包、检索软件包依赖性。

最安全最简单的方法是以root身份运行

pacman-optimize && sync

上述命令试图将所有小文件放在磁盘上同一个物理区域,以减少磁头移动。这种方法很安全,但不一定有效。其效果取决于你的文件系统、磁盘使用率、和磁盘碎片程度。

进一步的优化

ody 曾经发表了 a script on the forum 这篇文章,将目前的Pacman数据库替换成了一个回送文件系统(loopback filesystem),用来保证所有的小文件连续地存放在硬盘上。不少用户报告说会有很大的性能提升,但是也有问题出现。所以除非你是专业人士,否则不要这样做。

要使用ody的脚本,你必须确定内核添加了"loopback filesystem"支持(内核一般默认支持)。

加快下载速度

如果你的包下载速度变得极慢,首先确定你用的是那些镜像(mirrors)网站而不是ftp.archlinux.org,因为后者根据as of March 2007所述,被限速了。

可以通过各种下载工具而不是Pacman内置的下载方式,来改善Pacman的下载速度。

不论怎样,在做任何修改前,你必须确定拥有了最新版的Pacman:

pacman -Sy pacman

使用wget

对于需要更强大代理支持的用户来说,用wget比用Pacman自己的下载方式更加方便。

要使用 wget,首先使用pacman -S wget安装它,然后修改/etc/pacman.conf并在其中的[options]区段添加如下几行:

XferCommand = /usr/bin/wget -c --passive-ftp -c %u

除了将wget参数放在/etc/pacman.conf里,你也可以直接修改wget配置文件(全局文件是/etc/wgetrc,各个用户的文件是$HOME/.wgetrc)。

使用aria2

According to the aria2 website, aria2 is "a download utility with resuming and segmented downloading. Supports HTTP/HTTPS/FTP/BitTorrent/Metalink." This means that you can make several HTTP/FTP connections to an Arch mirror at the same time, which should result in an increase in download speeds.

Install it with pacman -S aria2 and then edit /etc/pacman.conf by adding the following line to the [option] section:

XferCommand = /usr/bin/aria2c -s 2 -m 2 -d / -o %o %u

Let's run over the options here:

  • /usr/bin/aria2c - the location of the aria2 application
  • -s 2 - use 2 concurrent connections (you can set this higher if you want, but it's not going to do a whole lot)
  • -m 2 - make 2 attempts to download the package per mirror
  • -o %o - output to the file pacman specifies
  • %u - download the file pacman specifies


Aria2 Mirror Script

This script will greatly improve the download speed for broadband users. It takes the download URL from pacman, then looks up the mirror list in /etc/pacman.d/mirrorlist and adds them all as mirrors to aria2. What ends up happening is aria2 connects to 10-20 servers (the default is 4 servers max) downloading from all of them at the same time, which should give anyone on broadband a huge boost in download speed. This should max out the download speed for most people.

Take note that you have to put 'exec' before /usr/bin/pacget in the XferCommand. This is needed so that when you terminate pacget or aria2 (with process id used by pacget), pacman would also terminate. This would prevent "inconvenience" because Pacman would not persist downloading a file when you tell it not to.

WARNING: You may experience a lot of problems when you become greedy. Just choose a handful of reliable servers (5 is already a lot) that syncs regularly with the master server. Do NOT choose out-of-date mirrors as these may cause problems like corrupted downloads, etc. Also, ftp.archlinux.org resolves to two IPs. You may want to choose only one of them and hard code ftp.archlinux.org and the chosen IP address to /etc/hosts.

WARNING: Needs testing with custom repos.

#!/bin/bash

# ------------ Begin Configuration ------------ #

# The log file
LOG=/var/log/pacget.log
# Number of connections per server (don't go beyond 2, please)
CONNECTIONS=2
# Max number of servers to download from (follows ordering in include file)
SERVERS=4
# Maximum download speed (0 = unrestricted)
MAX_SPEED=0
# Minimum download speed (0 = don't care)
MIN_SPEED=0
# Maximum tries per download
MAX_TRIES=2
# Server timeout period
TIMEOUT=15
# Passive FTP or not: 'yes' or 'no'
FTP_PASV="no"
# 'none' or 'prealloc'
FILE_ALLOC="none"
# Use color in messages
USE_COLOR="yes"

# ------------- End Configuration ------------- #

msg() {
  echo ""
  if [ "${USE_COLOR}" = "yes" ]; then
    echo -ne "   \033[1;34m->\033[1;0m \033[1;1m${1}\033[1;0m" >&2
  else
    echo -n "   -> ${1}" >&2
  fi
}

error() {
  if [ "$USE_COLOR" = "yes" ]; then
    echo -e "\033[1;31m==> ERROR:\033[1;0m \033[1;1m$1\033[1;0m" >&2
  else
    echo "==> ERROR: $1" >&2
  fi
}

ARIA2_BIN=$(which aria2c 2> /dev/null)

# ----- do some checks first -----
if [ ! -x "$ARIA2_BIN" ]; then
  error "aria2c was not found or isn't executable."
  exit 1
fi

if [ $# -ne 2 ]; then
  error "Incorrect number of arguments"
  exit 1
fi

filename=$(basename ${1})
server=${1%/${filename}}

# Determine which repo is being used
repo=$( awk -F'/' '$(NF-2)~/^(community|core|current|extra|testing|unstable)$/{print $(NF-2)}' <<<$server )
[ -z $repo ] && repo="custom"

# Override number of connections for db files
[[ ${filename} = *.db.tar.gz ]] && CONNECTIONS=1

# For db files, or when using a custom repo (which most likely doesn't have any mirror),
# use only the URL passed by pacman; Otherwise, extract the list of servers (from the include file of the repo) to download from
url=${1}
if ! [[ ${filename} = *.db.tar.gz || ${repo} = "custom" ]]; then
        mirrorlist=$( awk -F' *= *' '$0~"^\\["r"\\]",/Include *= */{l=$2} END{print l}' r=${repo} /etc/pacman.conf )
        [ -n mirrorlist ] && 
        url=$( sed -r '/^Server *= */!d; s/Server *= *//; s/\$repo'"/${repo}/; s:$:/${filename}:" ${mirrorlist}|head -n ${SERVERS} )
        #awk way of doing the same thing, 
        #url=$( awk -F' *= *' 'i<=n&&$1=="Server"{gsub(/\$repo/, r, $2);print $2"/"f;i++}' n=${SERVERS} r=${repo} f=${filename} ${mirrorlist} )
fi

# Passive FTP or not?
[ "${FTP_PASV}" = "yes" ] && OPT_FTP_PASV="--ftp-pasv" || OPT_FTP_PASV=""

msg "Downloading ${filename}"

cd /var/cache/pacman/pkg/

${ARIA2_BIN} --log=${LOG} --timeout=${TIMEOUT} --max-tries=${MAX_TRIES} --allow-overwrite=true \
             --split=${CONNECTIONS} ${OPT_FTP_PASV} --file-allocation=${FILE_ALLOC} \
             --lowest-speed-limit=${MIN_SPEED} --max-download-limit=${MAX_SPEED} \
             ${url} --out=${filename}.pacget && [ ! -f ${filename}.pacget.aria2 ] && mv ${filename}.pacget ${2} && chmod 644 ${2}

exit $?

Save this script as /usr/bin/pacget.

chmod 755 /usr/bin/pacget

This makes the script an executable

In /etc/pacman.conf the following needs to be added:

XferCommand = exec /usr/bin/pacget %u %o

PS: If you use ftp.archlinux.org as the first server listed in your include files (/etc/pacman.d/*), some problems may occur when the mirrors you are using have not yet synced. To make great use of this script, choose a mirror (that syncs in a timely manner) that is more appropriate for you, then put that on top of the server lists. This is to prevent downloading only from ftp.archlinux.org when the mirrors have not yet synced.

Using other applications

There are other downloading applications that you can use with Pacman. Here they are, and their associated XferCommand settings:

  • snarf: XferCommand = /usr/bin/snarf -N %u
  • lftp: XferCommand = /usr/bin/lftp -c pget %u