Difference between revisions of "Local Mirror (简体中文)"

From ArchWiki
Jump to: navigation, search
(FTP服务器配置)
Line 11: Line 11:
 
{{translateme}}
 
{{translateme}}
  
'''注意:''' 95%的用户不需要这么做。同步core和extra上的所有软件包会带来许多你所不需要的东西。Only follow these instructions if you are running a very large site of Arch machines or want to create an official mirror. Perhaps using a [[Network Shared Pacman Cache]] would serve you better.
+
'''注意:''' 95%的用户不需要这么做。同步core和extra上的所有软件包会带来许多你所不需要的东西。只有当你的站点运行着许多 Arch 系统时,或者想要创建一个官方镜像时,才有必要按照以下指导进行。也许使用 [[Network Shared Pacman Cache]] 会更好地满足你的需要。
  
'''NOTE:''' Due to traffic issues rsyncing from rsync.archlinux.org is allowed only for official mirrors. If you want to create an official mirror - write to the mailing list about this and entries in /etc/pacman.d/* will be added and your IP address will be allowed.
+
'''注意:''' 由于流量的问题,只允许官方镜像同步 rsync.archlinux.org。如果你想要建立一个官方镜像——发送邮件到邮件列表说明此事,之后 /etc/pacman.d/* 中的入口会被添加,你的 IP 地址也将被允许。
  
If you want to get a full mirror for personal use only, you may rsync from '''rsync://distro.ibiblio.org/distros/archlinux/'''
+
如果你只是用于个人使用,可以同步自 '''rsync://distro.ibiblio.org/distros/archlinux/'''
  
 
This document describes how to create a mirror on your local machine of all the packages and iso files on the Arch mirrors, how to update it using cron, how to serve the mirror with vsftpd, and how to set up pacman to use the local mirror.
 
This document describes how to create a mirror on your local machine of all the packages and iso files on the Arch mirrors, how to update it using cron, how to serve the mirror with vsftpd, and how to set up pacman to use the local mirror.

Revision as of 05:45, 24 January 2009

Template:I18n links start Template:I18n entry Template:I18n entry Template:I18n links end


Tango-preferences-desktop-locale.pngThis article or section needs to be translated.Tango-preferences-desktop-locale.png

Notes: please use the first argument of the template to provide more detailed indications. (Discuss in Talk:Local Mirror (简体中文)#)

注意: 95%的用户不需要这么做。同步core和extra上的所有软件包会带来许多你所不需要的东西。只有当你的站点运行着许多 Arch 系统时,或者想要创建一个官方镜像时,才有必要按照以下指导进行。也许使用 Network Shared Pacman Cache 会更好地满足你的需要。

注意: 由于流量的问题,只允许官方镜像同步 rsync.archlinux.org。如果你想要建立一个官方镜像——发送邮件到邮件列表说明此事,之后 /etc/pacman.d/* 中的入口会被添加,你的 IP 地址也将被允许。

如果你只是用于个人使用,可以同步自 rsync://distro.ibiblio.org/distros/archlinux/

This document describes how to create a mirror on your local machine of all the packages and iso files on the Arch mirrors, how to update it using cron, how to serve the mirror with vsftpd, and how to set up pacman to use the local mirror.

安装准备

首先,更新或是安装安装必要的工具:

pacman -Sy rsync vsftpd

现在,我们要创建一个用于同步镜像的新用户(没有登录特权),使用FTP获取文件。用户名“mirror”只是一个例子,你可以使用其他任何名称。请不要使用root或是其他能登录系统的帐号。安全起见,我们使用的帐号只能拥有尽可能少的权限。

useradd -m -s /bin/false mirror

现在,我们将继续创建镜像。

创建本地镜像目录

我们将使用 /home/mirror,刚才所创建的非特权用户的主目录,用于存储脚本,日志以及软件包。

首先要做的是在 /home/mirror 下创建几个目录 :

cd /home/mirror
sudo -u mirror mkdir {scripts,files,logs}

同步脚本

现在让我们用最喜爱的编辑器创建实际的 rsync 脚本, scripts/mirrorsync.sh

#!/bin/bash
#
# The script to sync a local mirror of the Arch Linux repositories and ISOs
#
# Copyright (C) 2007 Woody Gilk <woody@archlinux.org>
# Modifications by Dale Blount <dale@archlinux.org>
# and Roman Kyrylych <roman@archlinux.org>
# Licensed under the GNU GPL (version 2)

# Filesystem locations for the sync operations
SYNC_HOME="/home/mirror"
SYNC_LOGS="$SYNC_HOME/logs"
SYNC_FILES="$SYNC_HOME/files"
SYNC_LOCK="$SYNC_HOME/mirrorsync.lck"

# Select which repositories to sync
# Valid options are: core, extra, testing, community, iso
# Leave empty to sync a complete mirror
# SYNC_REPO=(core extra testing community iso)
SYNC_REPO=()

# Set the rsync server to use
# Only official public mirrors are allowed to use rsync.archlinux.org
# SYNC_SERVER=rsync.archlinux.org::ftp
SYNC_SERVER=distro.ibiblio.org::distros/archlinux

# Set the format of the log file name
# This example will output something like this: sync_20070201-8.log
LOG_FILE="pkgsync_$(date +%Y%m%d-%H).log"

# Do not edit the following lines, they protect the sync from running more than
# one instance at a time
if [ ! -d $SYNC_HOME ]; then
  echo "$SYNC_HOME does not exist, please create it, then run this script again."
  exit 1
fi

[ -f $SYNC_LOCK ] && exit 1
touch "$SYNC_LOCK"
# End of non-editable lines

# Create the log file and insert a timestamp
touch "$SYNC_LOGS/$LOG_FILE"
echo "=============================================" >> "$SYNC_LOGS/$LOG_FILE"
echo ">> Starting sync on $(date --rfc-3339=seconds)" >> "$SYNC_LOGS/$LOG_FILE"
echo ">> ---" >> "$SYNC_LOGS/$LOG_FILE"

if [ -z $SYNC_REPO ]; then
  # Sync a complete mirror
  rsync -rptLv --delete-after --delay-updates $SYNC_SERVER "$SYNC_FILES" >> "$SYNC_LOGS/$LOG_FILE"
else
  # Sync each of the repositories set in $SYNC_REPO
  for repo in ${SYNC_REPO[@]}; do
    repo=$(echo $repo | tr [:upper:] [:lower:])
    echo ">> Syncing $repo to $SYNC_FILES/$repo" >> "$SYNC_LOGS/$LOG_FILE"

    # If you only want to mirror i686 packages, you can add
    # " --exclude=os/x86_64" after "--delete-after"
    # 
    # If you only want to mirror x86_64 packages, use "--exclude=os/i686"
    # If you want both i686 and x86_64, leave the following line as it is
    #
    rsync -rptlv --safe-links --delete-after --delay-updates $SYNC_SERVER/$repo "$SYNC_FILES" >> "$SYNC_LOGS/$LOG_FILE"

    # Create $repo.lastsync file with timestamp like "2007-05-02 03:41:08+03:00"
    # which may be useful for users to know when the repository was last updated
    # date --rfc-3339=seconds > "$SYNC_FILES/$repo.lastsync"

    # Sleep 5 seconds after each repository to avoid too many concurrent connections
    # to rsync server if the TCP connection does not close in a timely manner
    sleep 5 
  done
fi

# Insert another timestamp and close the log file
echo ">> ---" >> "$SYNC_LOGS/$LOG_FILE"
echo ">> Finished sync on $(date --rfc-3339=seconds)" >> "$SYNC_LOGS/$LOG_FILE"
echo "=============================================" >> "$SYNC_LOGS/$LOG_FILE"
echo "" >> "$SYNC_LOGS/$LOG_FILE"

# Remove the lock file and exit
rm -f "$SYNC_LOCK"
exit 0

这并不是什么十分精美的,而只是一个能完成我们所需的稍微高级的bash脚本。让我们使其具有可执行权限。

chmod +x scripts/mirrorsync.sh

现在你已经有一个很容易修改的脚本。不过你可能不是很愿意每次都手动执行,我们可以设置一个cron任务来运行它。

在进行下一步前注意:你的 logs 目录将会逐渐增大。确保定期检查目录大小以防你的服务器因此崩溃。推荐设置 LogRotate 或者写一些清理脚本以处理。

运行 cron 任务

首先,确保具有必须的 cron 工具 (大多数的Archlinux都有):

pacman -S dcron

我们将用 crontab 运行 cron 任务。详情,参见 man crontab。用 crontab 而不是散落在 /etc/cron.* 各处的文件,来执行 sync 的好处是更高的安全性。它也使脚本运行有了更高级的控制。

以如下内容创建 scripts/mirror.cron

0 3 * * * /home/mirror/scripts/mirrorsync.sh

现在我们要激活我们的 crontab:

sudo -u mirror crontab scripts/mirror.cron

确认该 crontab 得到了我们的任务:

sudo -u mirror crontab -l

你将会看到 scripts/mirror.cron 的内容被显示出来。如果没有重新运行以上命令并检查。

这个 cron 设置将会在每天凌晨3点运行我们的 sync.sh 脚本。你可以随意调整这个时间,参见 http://www.adminschoice.com/docs/crontab.htm 了解更多 crontab 语法。

修改 cron 任务

如果你需要修改 mirror.cron,使用以下命令:

sudo -u mirror crontab -e

如果你手动修改了这个文件,使用以下命令更新 crontab:

sudo -u mirror crontab scripts/mirror.cron

现在我们开始设置 pacman 来使用这个本地镜像。

设置 pacman 使用本地镜像

如果只想在一台计算机上访问镜像,按照如下步骤进行。

单台计算机

注意: 如果你只是为一台计算机准备本地镜像,那就是毫无意义地占用了大量的带宽。为需要的人节约带宽吧。应用本节需同样遵循以下各节。

这种情况下你无需 vsftpd,因为我们将使用 file:// 定位符访问文件,而不是 ftp:// 定位符。

/etc/pacman.d/core 的开头,服务器列表的顶部添加如下行:

Server = file:///home/mirror/files/core/os/i686

可以将 /core/ 替换成你所镜像的软件库后添加 (例如:对于 community 库,该行应该像这样):

Server = file:///home/mirror/files/community/os/i686

如果你的系统是64位的,确保将 i686 改为 x86_64

多台计算机

这种方式同步允许多台计算机访问你的本地镜像。你同样能够使用这种方法同步到你的本地计算机(详情附后)。

FTP服务器配置

我们要做的第一步是配置 vsftpd。编辑 /etc/vsftpd.conf 成以下内容:

# vsftpd config file /etc/vsftpd.conf
#
# Setup for a secure anonymous FTP server
#
# Listen (non-xinetd) mode
listen=YES
# Use tcp_wrappers to control connections
tcp_wrappers=YES
# Use localtimes instead of GMT for files
use_localtime=YES
# Hide the true user/group ID of files
hide_ids=YES
# 
# Enable anonymous access (pacman requires this)
anonymous_enable=YES
# Use this user for anonymous logins
ftp_username=mirror
# Chroot directory for anonymous user
anon_root=/home/mirror/files
# Don't require a password for anonymous access (pacman requires this)
no_anon_password=YES
#
# User to run vsftpd as (same as ftp_username)
nopriv_user=mirror
# Enable recursive "ls" listing
ls_recurse_enable=YES
#
# Forcefully destroy sessions after X seconds of inactivity 
# (It is highly recommended to not set this above 300)
idle_session_timeout=120
# Forcefully stop sending data after X seconds of inactivity during a transfer
# (It is highly recommended to not set this higher than idle_session_timeout)
data_connection_timeout=30

如此设置 FTP server 将十分安全,且根据我们的需要量身定制。注意这样设置的服务器“不”要求口令,并且不应被用于公共网络(除非这是你想要的)。使用口令保护 FTP 服务器并使其仍能被 pacman 正常访问超出了本文档的讨论范围。

如果你将从外部连接到这台计算机,需要添加如下行到 /etc/hosts.allow 中:

vsftpd : ALL : ALL

注意这将允许所有人能下载这个镜像。如果你需要更严格地控制下载,并且不知道怎么做,参考 linux.about.com 的相关主题。

现在启动 vsftpd:

sudo /etc/rc.d/vsftpd start

如果 vsftpd 没有启动,检查 /etc/vsftpd.conf 文件中各选项设置是否正确。

Enabling the mirror for pacman

Now let's edit /etc/pacman.d/* files to use our shiny new mirror. Add the following line to the top of /etc/pacman.d/core, at the top of the Servers list:

Server = ftp://192.168.1.21/core/os/i686

Note that 192.168.1.21 is the IP address of my test machine. Your address will most likely be different. (Remember that you can get the current IP of an Arch box with ifconfig -a or ifconfig eth0.)

If you want to use this same mirror on the local machine, use the following Server line:

Server = ftp://localhost/core/os/i686

Repeat this same process for the other repositories you are mirroring. Non-local machines will need to use an IP address to access the repository. Also make sure that the machine serving the mirror has a static IP address.

Synchronizing for the first time

Here comes the pain! Run the following command to start the sync:

sudo -u mirror ./scripts/mirrorsync.sh

This won't give you any kind of output, but you probably want some. You can use something like this (correct for the name of log file) to monitor the sync progress:

tail -f logs/pkgsync_20070203-9.log

This process will usually take a few hours, depending on the speed of your internet connection and how many repositories you are mirroring. After the first sync, only new packages will be sync'ed, so it will be m