Difference between revisions of "Package Proxy Cache"

From ArchWiki
Jump to: navigation, search
m (Introduction)
(Some cosmetic changes (sentences start with an upper-case and end with a period, etc.))
Line 1: Line 1:
 
==Introduction==
 
==Introduction==
if you want to install the same arch packages over and over - like for testing AIF profiles - it could help if you wouldnt have to get the packages everytime from the internet. this arcticle shows you how to setup a [[Squid]] proxy that only caches arch packages and can be used with aif/pacman/wget/etc with minimal configuration on the client system.
+
If you want to install the same Arch packages over and over - like for testing AIF profiles - it could help if you wouldn't have to get the packages every time from the internet. This article shows you how to setup a [[Squid]] proxy that only caches arch packages and can be used with aif/pacman/wget/etc with minimal configuration on the client system.
  
{{Note|this document is still under testing!}}
+
{{Note|This document is still under testing!}}
  
 
==Install Squid==
 
==Install Squid==
Line 8: Line 8:
  
 
==Configure Squid==
 
==Configure Squid==
this is the minimum configuration to get squid cache arch packages.
+
This is the minimum configuration to get squid cache arch packages.
  
 
===Cache Rules===
 
===Cache Rules===
before defining these rules, remove/comment (if you dont need them) all the default refresh_patterns
+
Before defining these rules, remove/comment (if you don't need them) all the default refresh_patterns
 
{{File|name=/etc/squid/squid.conf (Line: ~2706)|content=refresh_pattern \.pkg\.tar\.  0      20%    4320      reload-into-ims
 
{{File|name=/etc/squid/squid.conf (Line: ~2706)|content=refresh_pattern \.pkg\.tar\.  0      20%    4320      reload-into-ims
 
refresh_pattern .              0      0%      0}}
 
refresh_pattern .              0      0%      0}}
that should define that *.pkg.tar.* gets cached, and anything else should not
+
That should define that *.pkg.tar.* gets cached, and anything else should not.
 
{{Tip|http://www.squid-cache.org/Doc/config/refresh_pattern}}
 
{{Tip|http://www.squid-cache.org/Doc/config/refresh_pattern}}
  
 
===Maximum Filesize===
 
===Maximum Filesize===
Objects larger than this size will NOT be saved on disk
+
Objects larger than this size will NOT be saved on disk:
 
{{File|name=/etc/squid/squid.conf (Line: ~1950)|content=maximum_object_size 256 MB}}
 
{{File|name=/etc/squid/squid.conf (Line: ~1950)|content=maximum_object_size 256 MB}}
 
{{Tip|http://www.squid-cache.org/Doc/config/maximum_object_size}}
 
{{Tip|http://www.squid-cache.org/Doc/config/maximum_object_size}}
  
 
===Cache Directory===
 
===Cache Directory===
Set the cache dir and its maximum size and subdirs
+
Set the cache dir and its maximum size and subdirs:
 
{{File|name=/etc/squid/squid.conf (Line: ~1911)|content=cache_dir aufs /var/cache/squid 10000 16 256}}
 
{{File|name=/etc/squid/squid.conf (Line: ~1911)|content=cache_dir aufs /var/cache/squid 10000 16 256}}
 
{{Tip|http://www.squid-cache.org/Doc/config/cache_dir}}
 
{{Tip|http://www.squid-cache.org/Doc/config/cache_dir}}
  
 
{{Note|
 
{{Note|
everytime (and after fresh install) when you change the cache_dir path, you need to (re)create this directory
+
Every time you change the cache_dir path (and after fresh install), you need to (re)create this directory:
 
{{cli|# squid -z}}
 
{{cli|# squid -z}}
and it could be helpful to check the config file before running
+
and it could be helpful to check the config file before running squid:
 
{{cli|# squid -k check}}
 
{{cli|# squid -k check}}
 
}}
 
}}
Line 36: Line 36:
 
==Start Squid==
 
==Start Squid==
 
{{cli|# /etc/rc.d/squid start}}
 
{{cli|# /etc/rc.d/squid start}}
or if squid is already running
+
or if squid is already running:
 
{{cli|# /etc/rc.d/squid restart}}
 
{{cli|# /etc/rc.d/squid restart}}
 
{{Note|
 
{{Note|
it could be helpful to check the config file before running
+
It could be helpful to check the config file before running:
 
{{cli|# squid -k check}}
 
{{cli|# squid -k check}}
 
}}
 
}}
  
==Follow Squid Access==
+
==Follow Squid access log==
to see the acces to squid
+
To see the access to squid:
 
{{cli|# tail -f /var/log/squid/access.log}}
 
{{cli|# tail -f /var/log/squid/access.log}}
you should see this for packages that are directed to original host
+
You should see this for packages that are directed to original host:
 
{{cli|...TCP_MISS/200...DIRECT...}}
 
{{cli|...TCP_MISS/200...DIRECT...}}
and for packages that are delirever from the cache
+
and for packages that are delivered from the cache:
 
{{cli|...TCP_HIT/200...NONE...}}
 
{{cli|...TCP_HIT/200...NONE...}}
  
 
==Manual Arch Install==
 
==Manual Arch Install==
before running /arch/setup, add variables for your proxy. to do so, run on the console
+
Before running /arch/setup, add variables for your proxy. To do so, run on the console:
 
{{cli|# export http_proxy&#61;'http://your_squid_machine_ip:3128'<br/># export ftp_proxy&#61;'ftp://your_squid_machine_ip:3128'}}
 
{{cli|# export http_proxy&#61;'http://your_squid_machine_ip:3128'<br/># export ftp_proxy&#61;'ftp://your_squid_machine_ip:3128'}}
  
now just use /arch/setup to normally install the system, and it should use your proxy. watch the squid logs to verify this.
+
Now just use /arch/setup to normally install the system, and it should use your proxy. Watch the squid logs to verify this.
  
{{Note|if you want to use the proxy in the installed system, you need to add the http_proxy and/or ftp_proxy variables in an appropiate place.}}
+
{{Note|If you want to use the proxy in the installed system, you need to add the http_proxy and/or ftp_proxy variables in an appropriate place.}}
  
 
==Automatic Arch Install==
 
==Automatic Arch Install==
Line 89: Line 89:
 
PARTITIONS='/dev/sda *:ext3'
 
PARTITIONS='/dev/sda *:ext3'
 
BLOCKDATA='/dev/sda1 raw no_label ext3;yes;/;target;no_opts;no_label;no_params'}}
 
BLOCKDATA='/dev/sda1 raw no_label ext3;yes;/;target;no_opts;no_label;no_params'}}
and install the system according to that profile
+
And install the system according to that profile:
 
{{cli|# aif -p automatic -c custom.aifprofile}}
 
{{cli|# aif -p automatic -c custom.aifprofile}}
now sit back, and enjoy the automation :)
+
Now sit back, and enjoy the automation :)
{{Tip| watch the squid logs, specially that on the second run you get TCP_HIT and NONE}}
+
{{Tip| Watch the squid logs, specially that on the second run you get TCP_HIT and NONE}}
and after the setup is done, the installed system should use your proxy for pacman/wget/etc.
+
After the setup is done, the installed system should use your proxy for pacman/wget/etc.

Revision as of 17:58, 24 May 2010

Introduction

If you want to install the same Arch packages over and over - like for testing AIF profiles - it could help if you wouldn't have to get the packages every time from the internet. This article shows you how to setup a Squid proxy that only caches arch packages and can be used with aif/pacman/wget/etc with minimal configuration on the client system.

Note: This document is still under testing!

Install Squid

Template:Cli

Configure Squid

This is the minimum configuration to get squid cache arch packages.

Cache Rules

Before defining these rules, remove/comment (if you don't need them) all the default refresh_patterns Template:File That should define that *.pkg.tar.* gets cached, and anything else should not.

Maximum Filesize

Objects larger than this size will NOT be saved on disk: Template:File

Cache Directory

Set the cache dir and its maximum size and subdirs: Template:File

Note:

Every time you change the cache_dir path (and after fresh install), you need to (re)create this directory: Template:Cli and it could be helpful to check the config file before running squid: Template:Cli

Start Squid

Template:Cli or if squid is already running: Template:Cli

Note:

It could be helpful to check the config file before running: Template:Cli

Follow Squid access log

To see the access to squid: Template:Cli You should see this for packages that are directed to original host: Template:Cli and for packages that are delivered from the cache: Template:Cli

Manual Arch Install

Before running /arch/setup, add variables for your proxy. To do so, run on the console: Template:Cli

Now just use /arch/setup to normally install the system, and it should use your proxy. Watch the squid logs to verify this.

Note: If you want to use the proxy in the installed system, you need to add the http_proxy and/or ftp_proxy variables in an appropriate place.

Automatic Arch Install

After login in the arch install image, setup your network (eg. `dhcpcd eth0`) and get this AIF profile on it.

Warning: This PARTITIONS AND FORMATS the defined device, so please go through the whole file before you run it!

Template:File And install the system according to that profile: Template:Cli Now sit back, and enjoy the automation :)

Tip: Watch the squid logs, specially that on the second run you get TCP_HIT and NONE

After the setup is done, the installed system should use your proxy for pacman/wget/etc.