Difference between revisions of "Package Proxy Cache"
(→Introduction: remove note about testing, im using this for days now :P) |
Kynikos.bot (talk | contribs) (Template:i18n is deprecated, use interlanguage links, see Help talk:I18n#"Dummy" interlanguage links and deprecation of Template:i18n) |
||
(5 intermediate revisions by 5 users not shown) | |||
Line 1: | Line 1: | ||
− | + | [[Category:Package management]] | |
If you want to install the same Arch packages over and over - like for testing AIF profiles - it could help if you wouldn't have to get the packages every time from the internet. This article shows you how to setup a [[Squid]] proxy that only caches arch packages and can be used with aif/pacman/wget/etc with minimal configuration on the client system. | If you want to install the same Arch packages over and over - like for testing AIF profiles - it could help if you wouldn't have to get the packages every time from the internet. This article shows you how to setup a [[Squid]] proxy that only caches arch packages and can be used with aif/pacman/wget/etc with minimal configuration on the client system. | ||
==Install Squid== | ==Install Squid== | ||
− | {{ | + | {{bc|# pacman -S squid}} |
==Configure Squid== | ==Configure Squid== | ||
Line 9: | Line 9: | ||
===Cache Rules=== | ===Cache Rules=== | ||
− | Before defining these rules, remove/comment (if you | + | Before defining these rules, remove/comment (if you do not need them) all the default refresh_patterns |
− | {{ | + | {{hc|/etc/squid/squid.conf |refresh_pattern \.pkg\.tar\. 0 20% 4320 reload-into-ims |
refresh_pattern . 0 0% 0}} | refresh_pattern . 0 0% 0}} | ||
That should define that *.pkg.tar.* gets cached, and anything else should not. | That should define that *.pkg.tar.* gets cached, and anything else should not. | ||
Line 17: | Line 17: | ||
===Maximum Filesize=== | ===Maximum Filesize=== | ||
Objects larger than this size will NOT be saved on disk: | Objects larger than this size will NOT be saved on disk: | ||
− | {{ | + | {{hc|/etc/squid/squid.conf |maximum_object_size 256 MB}} |
{{Tip|http://www.squid-cache.org/Doc/config/maximum_object_size}} | {{Tip|http://www.squid-cache.org/Doc/config/maximum_object_size}} | ||
===Cache Directory=== | ===Cache Directory=== | ||
Set the cache dir and its maximum size and subdirs: | Set the cache dir and its maximum size and subdirs: | ||
− | {{ | + | {{hc|/etc/squid/squid.conf |cache_dir aufs /var/cache/squid 10000 16 256}} |
{{Tip|http://www.squid-cache.org/Doc/config/cache_dir}} | {{Tip|http://www.squid-cache.org/Doc/config/cache_dir}} | ||
===Shutdown Lifetime=== | ===Shutdown Lifetime=== | ||
Time to wait until all active client sockets are closed: | Time to wait until all active client sockets are closed: | ||
− | {{ | + | {{hc|/etc/squid/squid.conf |shutdown_lifetime 1 seconds }} |
{{Tip|http://www.squid-cache.org/Doc/config/shutdown_lifetime}} | {{Tip|http://www.squid-cache.org/Doc/config/shutdown_lifetime}} | ||
{{Note| | {{Note| | ||
Every time you change the cache_dir path (and after fresh install), you need to (re)create this directory: | Every time you change the cache_dir path (and after fresh install), you need to (re)create this directory: | ||
− | {{ | + | {{bc|# squid -z}} |
and it could be helpful to check the config file before running squid: | and it could be helpful to check the config file before running squid: | ||
− | {{ | + | {{bc|# squid -k check}} |
}} | }} | ||
==Start Squid== | ==Start Squid== | ||
− | {{ | + | {{bc|# /etc/rc.d/squid start}} |
or if squid is already running: | or if squid is already running: | ||
− | {{ | + | {{bc|# /etc/rc.d/squid restart}} |
{{Note| | {{Note| | ||
It could be helpful to check the config file before running: | It could be helpful to check the config file before running: | ||
− | {{ | + | {{bc|# squid -k check}} |
}} | }} | ||
==Follow Squid access log== | ==Follow Squid access log== | ||
To see the access to squid: | To see the access to squid: | ||
− | {{ | + | {{bc|# tail -f /var/log/squid/access.log}} |
You should see this for packages that are directed to original host: | You should see this for packages that are directed to original host: | ||
− | {{ | + | {{bc|...TCP_MISS/200...DIRECT...}} |
and for packages that are delivered from the cache: | and for packages that are delivered from the cache: | ||
− | {{ | + | {{bc|...TCP_HIT/200...NONE...}} |
==Manual Arch Install== | ==Manual Arch Install== | ||
Before running /arch/setup, add variables for your proxy. To do so, run on the console: | Before running /arch/setup, add variables for your proxy. To do so, run on the console: | ||
− | {{ | + | {{bc|# export http_proxy='http://your_squid_machine_ip:3128'<br/># export ftp_proxy='ftp://your_squid_machine_ip:3128'}} |
Now just use /arch/setup to normally install the system, and it should use your proxy. Watch the squid logs to verify this. | Now just use /arch/setup to normally install the system, and it should use your proxy. Watch the squid logs to verify this. | ||
Line 65: | Line 65: | ||
After login in the arch install image, setup your network (eg. `dhcpcd eth0`) and get this AIF profile on it. | After login in the arch install image, setup your network (eg. `dhcpcd eth0`) and get this AIF profile on it. | ||
{{Warning|This PARTITIONS AND FORMATS the defined device, so please go through the whole file before you run it!}} | {{Warning|This PARTITIONS AND FORMATS the defined device, so please go through the whole file before you run it!}} | ||
− | {{ | + | {{hc|custom.aifprofile|2= |
export ftp_proxy='ftp://your_squid_machine_ip:3128' | export ftp_proxy='ftp://your_squid_machine_ip:3128' | ||
export http_proxy='http://your_squid_machine_ip:3128' | export http_proxy='http://your_squid_machine_ip:3128' | ||
Line 90: | Line 90: | ||
GRUB_DEVICE=/dev/sda | GRUB_DEVICE=/dev/sda | ||
− | PARTITIONS='/dev/sda *: | + | PARTITIONS='/dev/sda *:ext4' |
− | BLOCKDATA='/dev/sda1 raw no_label | + | BLOCKDATA='/dev/sda1 raw no_label ext4;yes;/;target;no_opts;no_label;no_params'}} |
And install the system according to that profile: | And install the system according to that profile: | ||
− | {{ | + | {{bc|# aif -p automatic -c custom.aifprofile}} |
Now sit back, and enjoy the automation :) | Now sit back, and enjoy the automation :) | ||
{{Tip| Watch the squid logs, specially that on the second run you get TCP_HIT and NONE}} | {{Tip| Watch the squid logs, specially that on the second run you get TCP_HIT and NONE}} | ||
After the setup is done, the installed system should use your proxy for pacman/wget/etc. | After the setup is done, the installed system should use your proxy for pacman/wget/etc. |
Revision as of 14:25, 13 June 2012
If you want to install the same Arch packages over and over - like for testing AIF profiles - it could help if you wouldn't have to get the packages every time from the internet. This article shows you how to setup a Squid proxy that only caches arch packages and can be used with aif/pacman/wget/etc with minimal configuration on the client system.
Contents
Install Squid
# pacman -S squid
Configure Squid
This is the minimum configuration to get squid cache arch packages.
Cache Rules
Before defining these rules, remove/comment (if you do not need them) all the default refresh_patterns
/etc/squid/squid.conf
refresh_pattern \.pkg\.tar\. 0 20% 4320 reload-into-ims refresh_pattern . 0 0% 0
That should define that *.pkg.tar.* gets cached, and anything else should not.
Maximum Filesize
Objects larger than this size will NOT be saved on disk:
/etc/squid/squid.conf
maximum_object_size 256 MB
Cache Directory
Set the cache dir and its maximum size and subdirs:
/etc/squid/squid.conf
cache_dir aufs /var/cache/squid 10000 16 256
Shutdown Lifetime
Time to wait until all active client sockets are closed:
/etc/squid/squid.conf
shutdown_lifetime 1 seconds
Every time you change the cache_dir path (and after fresh install), you need to (re)create this directory:
# squid -z
and it could be helpful to check the config file before running squid:
# squid -k check
Start Squid
# /etc/rc.d/squid start
or if squid is already running:
# /etc/rc.d/squid restart
It could be helpful to check the config file before running:
# squid -k check
Follow Squid access log
To see the access to squid:
# tail -f /var/log/squid/access.log
You should see this for packages that are directed to original host:
...TCP_MISS/200...DIRECT...
and for packages that are delivered from the cache:
...TCP_HIT/200...NONE...
Manual Arch Install
Before running /arch/setup, add variables for your proxy. To do so, run on the console:
# export http_proxy='http://your_squid_machine_ip:3128'
# export ftp_proxy='ftp://your_squid_machine_ip:3128'
Now just use /arch/setup to normally install the system, and it should use your proxy. Watch the squid logs to verify this.
Automatic Arch Install
After login in the arch install image, setup your network (eg. `dhcpcd eth0`) and get this AIF profile on it.
custom.aifprofile
export ftp_proxy='ftp://your_squid_machine_ip:3128' export http_proxy='http://your_squid_machine_ip:3128' SOURCE=net FILE_URL='http://mirrors.kernel.org/archlinux/$repo/os/i686' SYNC_URL='http://mirrors.kernel.org/archlinux/$repo/os/i686' HARDWARECLOCK=localtime TIMEZONE=Europe/Berlin RUNTIME_REPOSITORIES= RUNTIME_PACKAGES= TARGET_GROUPS=base TARGET_PACKAGES_EXCLUDE= #TARGET_PACKAGES='xorg gnome ttf-dejavu' worker_configure_system () { prefill_configs echo "export http_proxy=$http_proxy" >> /mnt/etc/profile.d/proxy.sh echo "export ftp_proxy=$ftp_proxy" >> /mnt/etc/profile.d/proxy.sh chmod +x /mnt/etc/profile.d/proxy.sh sed -i 's/^HOSTNAME="myhost"/HOSTNAME="foohost"/' $var_TARGET_DIR/etc/rc.conf } GRUB_DEVICE=/dev/sda PARTITIONS='/dev/sda *:ext4' BLOCKDATA='/dev/sda1 raw no_label ext4;yes;/;target;no_opts;no_label;no_params'
And install the system according to that profile:
# aif -p automatic -c custom.aifprofile
Now sit back, and enjoy the automation :)
After the setup is done, the installed system should use your proxy for pacman/wget/etc.