https://wiki.archlinux.org/api.php?action=feedcontributions&user=Makan&feedformat=atomArchWiki - User contributions [en]2024-03-29T06:15:28ZUser contributionsMediaWiki 1.41.0https://wiki.archlinux.org/index.php?title=Local_mirror&diff=146169Local mirror2011-06-15T04:51:18Z<p>Makan: /* Client Configuration */</p>
<hr />
<div>[[Category:Package management (English)]]<br />
{{i18n|Local Mirror}}<br />
<br />
{{Warning| If you want to create an official mirror see [http://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors this page].}}<br />
<br />
==STOP==<br />
<br />
{{Warning|It is generally frowned upon to create a local mirror due the bandwidth that is required. One of the alternatives will likely fulfill your needs. Please look at the alternatives below.}}<br />
<br />
====Alternatives:====<br />
*[[Network Shared Pacman Cache]]<br />
<br />
==Local Mirror==<br />
<br />
===Things to keep in mind:===<br />
* Bandwidth is not free for the mirrors. They must pay for all the data they serve you<br />
** This still applies although you pay your ISP<br />
* There are many packages that will be downloaded that you will likely never use<br />
* Mirror operators will much prefer you to download only the packages you need<br />
* Really please look at the alternatives above<br />
<br />
'''If you are absolutely certain that a local mirror is the only sensible solution, then follow the pointers below.'''<br />
<br />
===Server Configuration===<br />
====Building Rsync Command====<br />
* Use the rsync arguments from [http://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors DeveloperWiki:NewMirrors]<br />
* Select a server from the above article<br />
* Exclude folder/files you don't want by including <tt>--exclude-from="/path/to/exclude.txt"</tt> in the rsync arguments. Example contents might include:<br />
<pre><br />
iso<br />
<br />
#Exclude i686 Packages<br />
*/os/i686<br />
pool/*/*-i686.pkg.tar.xz<br />
pool/*/*-i686.pkg.tar.gz<br />
<br />
#Exclude x86_64 Packages<br />
*/os/x86_64<br />
pool/*/*-x86_64.pkg.tar.xz<br />
pool/*/*-x86_64.pkg.tar.gz<br />
</pre><br />
<br />
* All packages reside in the pool directory. Symlinks are then created from pool to core/extra/testing/etc..<br />
** As of 9/21/2010 this migration is not yet complete.<br />
*** There may be actual packages, instead of symlinks, in ${repo}/os/${arch}<br />
* Exclude any top-level directories that you do not need<br />
<br />
Example: <tt>rsync ''$rsync_arguments'' --exclude="/path/to/exclude.txt" ''rsync://example.com/'' /path/to/destination</tt><br />
<br />
====Example Script====<br />
<br />
{{Warning|DO NOT USE THIS SCRIPT UNLESS YOU HAVE READ WARNINGS AT THE START OF THIS ARTICLE}}<br />
{{Warning|Only use this script to sync Core/Extra/Community! If you need Testing, gnome-unstable or any other repo, use rsync --exclude instead!}}<br />
<br />
Yes, this script is partially broken '''ON PURPOSE''' to avoid people doing copy-and-paste to create their own mirror. It should be easy to fix if you REALLY want a mirror.<br />
<br />
#!/bin/bash<br />
<br />
#################################################################################################<br />
### It is generally frowned upon to create a local mirror due the bandwidth that is required.<br />
### One of the alternatives will likely fulfill your needs.<br />
### REMEMBER:<br />
### * Bandwidth is not free for the mirrors. They must pay for all the data they serve you<br />
### => This still applies although you pay your ISP <br />
### => There are many packages that will be downloaded that you will likely never use<br />
### => Mirror operators will much prefer you to download only the packages you need<br />
### * Really please look at the alternatives on this page:<br />
### http://wiki.archlinux.org/index.php?title=Local_Mirror<br />
### If you are ABSOLUTELY CERTAIN that a local mirror is the only sensible solution, then this<br />
### script will get you on your way to creating it. <br />
#################################################################################################<br />
<br />
# Configuration<br />
SOURCE='rsync://mirror.example.com/archlinux'<br />
DEST='/srv/mirrors/archlinux'<br />
BW_LIMIT='500'<br />
REPOS='core extra'<br />
RSYNC_OPTS="-rtlHq --delete-after --delay-updates --copy-links --safe-links --max-delete=1000 --bwlimit=${BW_LIMIT} --delete-excluded --exclude=.*"<br />
LCK_FLE='/var/run/repo-sync.lck'<br />
<br />
# Make sure only 1 instance runs<br />
if [ -e "$LCK_FLE" ] ; then<br />
OTHER_PID=`/bin/cat $LCK_FLE`<br />
echo "Another instance already running: $OTHER_PID"<br />
exit 1<br />
fi<br />
echo $$ > "$LCK_FLE"<br />
<br />
for REPO in $REPOS ; do<br />
echo "Syncing $REPO"<br />
echo /usr/bin/rsync $RSYNC_OPTS ${SOURCE}/${REPO} ${DEST}<br />
done<br />
<br />
# Cleanup<br />
/bin/rm -f "$LCK_FLE"<br />
<br />
exit 0<br />
<br />
====Serving====<br />
*HTTP (LAN)<br />
**[[LAMP]]<br />
**[[Lighttpd]]<br />
*FTP (LAN)<br />
**[[vsftpd]]<br />
*Physical Media<br />
**Flash Drive<br />
**External HD<br />
<br />
===Client Configuration===<br />
*Add the proper Server= variable in /etc/pacman.d/mirrorlist<br />
*For physical media (such as flash drive) the following can be used: Server = file:///mnt/media/repo/$repo/os/$arch (''where /mnt/media/repo is directory where local mirror located'')</div>Makanhttps://wiki.archlinux.org/index.php?title=Local_mirror&diff=146168Local mirror2011-06-15T04:50:41Z<p>Makan: /* Client Configuration */</p>
<hr />
<div>[[Category:Package management (English)]]<br />
{{i18n|Local Mirror}}<br />
<br />
{{Warning| If you want to create an official mirror see [http://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors this page].}}<br />
<br />
==STOP==<br />
<br />
{{Warning|It is generally frowned upon to create a local mirror due the bandwidth that is required. One of the alternatives will likely fulfill your needs. Please look at the alternatives below.}}<br />
<br />
====Alternatives:====<br />
*[[Network Shared Pacman Cache]]<br />
<br />
==Local Mirror==<br />
<br />
===Things to keep in mind:===<br />
* Bandwidth is not free for the mirrors. They must pay for all the data they serve you<br />
** This still applies although you pay your ISP<br />
* There are many packages that will be downloaded that you will likely never use<br />
* Mirror operators will much prefer you to download only the packages you need<br />
* Really please look at the alternatives above<br />
<br />
'''If you are absolutely certain that a local mirror is the only sensible solution, then follow the pointers below.'''<br />
<br />
===Server Configuration===<br />
====Building Rsync Command====<br />
* Use the rsync arguments from [http://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors DeveloperWiki:NewMirrors]<br />
* Select a server from the above article<br />
* Exclude folder/files you don't want by including <tt>--exclude-from="/path/to/exclude.txt"</tt> in the rsync arguments. Example contents might include:<br />
<pre><br />
iso<br />
<br />
#Exclude i686 Packages<br />
*/os/i686<br />
pool/*/*-i686.pkg.tar.xz<br />
pool/*/*-i686.pkg.tar.gz<br />
<br />
#Exclude x86_64 Packages<br />
*/os/x86_64<br />
pool/*/*-x86_64.pkg.tar.xz<br />
pool/*/*-x86_64.pkg.tar.gz<br />
</pre><br />
<br />
* All packages reside in the pool directory. Symlinks are then created from pool to core/extra/testing/etc..<br />
** As of 9/21/2010 this migration is not yet complete.<br />
*** There may be actual packages, instead of symlinks, in ${repo}/os/${arch}<br />
* Exclude any top-level directories that you do not need<br />
<br />
Example: <tt>rsync ''$rsync_arguments'' --exclude="/path/to/exclude.txt" ''rsync://example.com/'' /path/to/destination</tt><br />
<br />
====Example Script====<br />
<br />
{{Warning|DO NOT USE THIS SCRIPT UNLESS YOU HAVE READ WARNINGS AT THE START OF THIS ARTICLE}}<br />
{{Warning|Only use this script to sync Core/Extra/Community! If you need Testing, gnome-unstable or any other repo, use rsync --exclude instead!}}<br />
<br />
Yes, this script is partially broken '''ON PURPOSE''' to avoid people doing copy-and-paste to create their own mirror. It should be easy to fix if you REALLY want a mirror.<br />
<br />
#!/bin/bash<br />
<br />
#################################################################################################<br />
### It is generally frowned upon to create a local mirror due the bandwidth that is required.<br />
### One of the alternatives will likely fulfill your needs.<br />
### REMEMBER:<br />
### * Bandwidth is not free for the mirrors. They must pay for all the data they serve you<br />
### => This still applies although you pay your ISP <br />
### => There are many packages that will be downloaded that you will likely never use<br />
### => Mirror operators will much prefer you to download only the packages you need<br />
### * Really please look at the alternatives on this page:<br />
### http://wiki.archlinux.org/index.php?title=Local_Mirror<br />
### If you are ABSOLUTELY CERTAIN that a local mirror is the only sensible solution, then this<br />
### script will get you on your way to creating it. <br />
#################################################################################################<br />
<br />
# Configuration<br />
SOURCE='rsync://mirror.example.com/archlinux'<br />
DEST='/srv/mirrors/archlinux'<br />
BW_LIMIT='500'<br />
REPOS='core extra'<br />
RSYNC_OPTS="-rtlHq --delete-after --delay-updates --copy-links --safe-links --max-delete=1000 --bwlimit=${BW_LIMIT} --delete-excluded --exclude=.*"<br />
LCK_FLE='/var/run/repo-sync.lck'<br />
<br />
# Make sure only 1 instance runs<br />
if [ -e "$LCK_FLE" ] ; then<br />
OTHER_PID=`/bin/cat $LCK_FLE`<br />
echo "Another instance already running: $OTHER_PID"<br />
exit 1<br />
fi<br />
echo $$ > "$LCK_FLE"<br />
<br />
for REPO in $REPOS ; do<br />
echo "Syncing $REPO"<br />
echo /usr/bin/rsync $RSYNC_OPTS ${SOURCE}/${REPO} ${DEST}<br />
done<br />
<br />
# Cleanup<br />
/bin/rm -f "$LCK_FLE"<br />
<br />
exit 0<br />
<br />
====Serving====<br />
*HTTP (LAN)<br />
**[[LAMP]]<br />
**[[Lighttpd]]<br />
*FTP (LAN)<br />
**[[vsftpd]]<br />
*Physical Media<br />
**Flash Drive<br />
**External HD<br />
<br />
===Client Configuration===<br />
*Add the proper Server= variable in /etc/pacman.d/mirrorlist<br />
*For physical media (such as flash drive) the following can be used: Server = file:///mnt/media/repo/$repo/os/$arch (where /mnt/media/repo is directory where local mirror located)</div>Makanhttps://wiki.archlinux.org/index.php?title=Local_mirror&diff=146167Local mirror2011-06-15T04:49:43Z<p>Makan: /* Client Configuration */</p>
<hr />
<div>[[Category:Package management (English)]]<br />
{{i18n|Local Mirror}}<br />
<br />
{{Warning| If you want to create an official mirror see [http://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors this page].}}<br />
<br />
==STOP==<br />
<br />
{{Warning|It is generally frowned upon to create a local mirror due the bandwidth that is required. One of the alternatives will likely fulfill your needs. Please look at the alternatives below.}}<br />
<br />
====Alternatives:====<br />
*[[Network Shared Pacman Cache]]<br />
<br />
==Local Mirror==<br />
<br />
===Things to keep in mind:===<br />
* Bandwidth is not free for the mirrors. They must pay for all the data they serve you<br />
** This still applies although you pay your ISP<br />
* There are many packages that will be downloaded that you will likely never use<br />
* Mirror operators will much prefer you to download only the packages you need<br />
* Really please look at the alternatives above<br />
<br />
'''If you are absolutely certain that a local mirror is the only sensible solution, then follow the pointers below.'''<br />
<br />
===Server Configuration===<br />
====Building Rsync Command====<br />
* Use the rsync arguments from [http://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors DeveloperWiki:NewMirrors]<br />
* Select a server from the above article<br />
* Exclude folder/files you don't want by including <tt>--exclude-from="/path/to/exclude.txt"</tt> in the rsync arguments. Example contents might include:<br />
<pre><br />
iso<br />
<br />
#Exclude i686 Packages<br />
*/os/i686<br />
pool/*/*-i686.pkg.tar.xz<br />
pool/*/*-i686.pkg.tar.gz<br />
<br />
#Exclude x86_64 Packages<br />
*/os/x86_64<br />
pool/*/*-x86_64.pkg.tar.xz<br />
pool/*/*-x86_64.pkg.tar.gz<br />
</pre><br />
<br />
* All packages reside in the pool directory. Symlinks are then created from pool to core/extra/testing/etc..<br />
** As of 9/21/2010 this migration is not yet complete.<br />
*** There may be actual packages, instead of symlinks, in ${repo}/os/${arch}<br />
* Exclude any top-level directories that you do not need<br />
<br />
Example: <tt>rsync ''$rsync_arguments'' --exclude="/path/to/exclude.txt" ''rsync://example.com/'' /path/to/destination</tt><br />
<br />
====Example Script====<br />
<br />
{{Warning|DO NOT USE THIS SCRIPT UNLESS YOU HAVE READ WARNINGS AT THE START OF THIS ARTICLE}}<br />
{{Warning|Only use this script to sync Core/Extra/Community! If you need Testing, gnome-unstable or any other repo, use rsync --exclude instead!}}<br />
<br />
Yes, this script is partially broken '''ON PURPOSE''' to avoid people doing copy-and-paste to create their own mirror. It should be easy to fix if you REALLY want a mirror.<br />
<br />
#!/bin/bash<br />
<br />
#################################################################################################<br />
### It is generally frowned upon to create a local mirror due the bandwidth that is required.<br />
### One of the alternatives will likely fulfill your needs.<br />
### REMEMBER:<br />
### * Bandwidth is not free for the mirrors. They must pay for all the data they serve you<br />
### => This still applies although you pay your ISP <br />
### => There are many packages that will be downloaded that you will likely never use<br />
### => Mirror operators will much prefer you to download only the packages you need<br />
### * Really please look at the alternatives on this page:<br />
### http://wiki.archlinux.org/index.php?title=Local_Mirror<br />
### If you are ABSOLUTELY CERTAIN that a local mirror is the only sensible solution, then this<br />
### script will get you on your way to creating it. <br />
#################################################################################################<br />
<br />
# Configuration<br />
SOURCE='rsync://mirror.example.com/archlinux'<br />
DEST='/srv/mirrors/archlinux'<br />
BW_LIMIT='500'<br />
REPOS='core extra'<br />
RSYNC_OPTS="-rtlHq --delete-after --delay-updates --copy-links --safe-links --max-delete=1000 --bwlimit=${BW_LIMIT} --delete-excluded --exclude=.*"<br />
LCK_FLE='/var/run/repo-sync.lck'<br />
<br />
# Make sure only 1 instance runs<br />
if [ -e "$LCK_FLE" ] ; then<br />
OTHER_PID=`/bin/cat $LCK_FLE`<br />
echo "Another instance already running: $OTHER_PID"<br />
exit 1<br />
fi<br />
echo $$ > "$LCK_FLE"<br />
<br />
for REPO in $REPOS ; do<br />
echo "Syncing $REPO"<br />
echo /usr/bin/rsync $RSYNC_OPTS ${SOURCE}/${REPO} ${DEST}<br />
done<br />
<br />
# Cleanup<br />
/bin/rm -f "$LCK_FLE"<br />
<br />
exit 0<br />
<br />
====Serving====<br />
*HTTP (LAN)<br />
**[[LAMP]]<br />
**[[Lighttpd]]<br />
*FTP (LAN)<br />
**[[vsftpd]]<br />
*Physical Media<br />
**Flash Drive<br />
**External HD<br />
<br />
===Client Configuration===<br />
*Add the proper Server= variable in /etc/pacman.d/mirrorlist<br />
*For physical media (such as flash drive) the following can be used: Server = file:///mnt/media/repo/$repo/os/$arch<br />
where /mnt/media/repo is directory where local mirror located</div>Makan