Difference between revisions of "Squid"

From ArchWiki
Jump to: navigation, search
m (codeline -> ic)
m (Updating the Iptables section with more info.)
(42 intermediate revisions by 16 users not shown)
Line 1: Line 1:
[[Category:Networking (English)]]
+
[[Category:Security]]
[[Category:Security (English)]]
+
[[Category:Proxy servers]]
{{i18n|Squid}}
+
[[ru:Squid]]
 +
[[zh-CN:Squid]]
 
{{poor writing}}
 
{{poor writing}}
  
 
From the squid [http://www.squid-cache.org website]:
 
From the squid [http://www.squid-cache.org website]:
  
:::''Squid is a caching proxy for the Web supporting HTTP, HTTPS, FTP, and more. It reduces bandwidth and improves response times by caching and reusing frequently-requested web pages. Squid has extensive access controls and makes a great server accelerator. It runs on Unix and Windows and is licensed under the GNU GPL.''
+
:''Squid is a caching proxy for the Web supporting HTTP, HTTPS, FTP, and more. It reduces bandwidth and improves response times by caching and reusing frequently-requested web pages. Squid has extensive access controls and makes a great server accelerator. It runs on Unix and Windows and is licensed under the GNU GPL.''
  
 
While squid works wonderfully in large corporations and schools, it can also benefit the home user too. However, if you're looking for a more lightweight single-user proxy, you should try [[Polipo]].
 
While squid works wonderfully in large corporations and schools, it can also benefit the home user too. However, if you're looking for a more lightweight single-user proxy, you should try [[Polipo]].
Line 12: Line 13:
 
== Installation ==
 
== Installation ==
  
pacman -S squid
+
[[pacman|Install]] {{Pkg|squid}} available in the [[Official Repositories]].
  
 
== Configuration ==
 
== Configuration ==
  
By default, the cache directories will be created in /var/cache/squid, and the appropriate permissions set up for those directories. However, for greater control, we need to delve into {{Ic|/etc/squid/squid.conf}}.
+
By default, the cache directories will be created in {{ic|/var/cache/squid}}, and the appropriate permissions set up for those directories. However, for greater control, we need to delve into {{Ic|/etc/squid/squid.conf}}.
  
 
Everything is well commented, but if you want to strip the comments out you should run:
 
Everything is well commented, but if you want to strip the comments out you should run:
Line 33: Line 34:
 
*{{Ic|cache_mgr}} - This is the email address of the cache manager.
 
*{{Ic|cache_mgr}} - This is the email address of the cache manager.
 
  cache_mgr squid.admin@example.com
 
  cache_mgr squid.admin@example.com
*{{Ic|shutdown_lifetime}} - Specifies how long Squid should wait when its rc.d script is asked to stop. If you're running squid on your desktop PC, you may want to set this to something short.
+
*{{Ic|shutdown_lifetime}} - Specifies how long Squid should wait when its service is asked to stop. If you're running squid on your desktop PC, you may want to set this to something short.
 
  shutdown_lifetime 10 seconds
 
  shutdown_lifetime 10 seconds
 
*{{Ic|cache_mem}} - This is how much memory you want Squid to use to keep objects in memory rather than writing them to disk. Squid's total memory usage will exceed this! By default this is 8MB, so you might want to increase it if you have lots of RAM available.
 
*{{Ic|cache_mem}} - This is how much memory you want Squid to use to keep objects in memory rather than writing them to disk. Squid's total memory usage will exceed this! By default this is 8MB, so you might want to increase it if you have lots of RAM available.
Line 40: Line 41:
 
  visible_hostname cerberus
 
  visible_hostname cerberus
 
*{{Ic|cache_peer}} - If you want your Squid to go through another proxy server, rather than directly out to the Internet, you need to specify it here.
 
*{{Ic|cache_peer}} - If you want your Squid to go through another proxy server, rather than directly out to the Internet, you need to specify it here.
 +
*{{Ic|login}} - Use this option if the parent proxy requires authentication.
 
*{{Ic|never_direct}} - Tells the cache to never go direct to the internet to retrieve a page. You will want this if you have set the option above.
 
*{{Ic|never_direct}} - Tells the cache to never go direct to the internet to retrieve a page. You will want this if you have set the option above.
  cache_peer 10.1.1.100 parent 8080 0 no-query default
+
  cache_peer 10.1.1.100 parent 8080 0 no-query default login=user:password
 
  never_direct allow all
 
  never_direct allow all
 
*{{Ic|maximum_object_size}} - The largest size of a cached object. By default this is small (256KB I think), so if you have a lot of disk space you will want to increase the size of it to something reasonable.
 
*{{Ic|maximum_object_size}} - The largest size of a cached object. By default this is small (256KB I think), so if you have a lot of disk space you will want to increase the size of it to something reasonable.
 
  maximum_object_size 10 MB
 
  maximum_object_size 10 MB
 +
{{Note|After defining a new cache_dir it maybe necessary to initialize the caches directory structure with this command: <code>squid -zN</code> -z for Create missing swap directories and -N for No daemon mode. }}
 
*{{Ic|cache_dir}} - This is your cache directory, where all the cached files are stored. There are many options here, but the format should generally go like:
 
*{{Ic|cache_dir}} - This is your cache directory, where all the cached files are stored. There are many options here, but the format should generally go like:
  cache_dir diskd <directory> <size in MB> 16 256
+
  cache_dir <storage type> <directory> <size in MB> 16 256
 
So, in the case of a school's internet proxy:
 
So, in the case of a school's internet proxy:
 
  cache_dir diskd /cache0 200000 16 256
 
  cache_dir diskd /cache0 200000 16 256
 
If you change the cache directory from defaults, you must set the correct permissions on the cache directory before starting Squid, else it won't be able to create its cache directories and will fail to start.
 
If you change the cache directory from defaults, you must set the correct permissions on the cache directory before starting Squid, else it won't be able to create its cache directories and will fail to start.
 +
 +
== Accessing services on local hostnames ==
 +
 +
If you plan to access web servers on the LAN using hostnames that are not fully-defined (e.g. {{ic|http://mywebapp}}), you may need to enable the {{ic|dns_defnames}} option.  Without this option, Squid will make a DNS request for the hostname verbatim ({{ic|mywebapp}}), which may fail, depending on your LAN's DNS setup.  With the option enabled, Squid will append any domain configured in {{ic|/etc/resolv.conf}} when making the request (e.g. {{ic|mywebapp.company.local}}).
 +
 +
{{bc|
 +
dns_defnames on
 +
}}
  
 
== Starting ==
 
== Starting ==
  
 
Once you have finished your configuration, you should check that your configuration file is correct:
 
Once you have finished your configuration, you should check that your configuration file is correct:
  squid -k check
+
  # squid -k check
 
Then create your cache directories:
 
Then create your cache directories:
  squid -z
+
  # squid -z
 
Then you can start Squid!
 
Then you can start Squid!
  /etc/rc.d/squid start
+
  # systemctl start squid
  
Don't forget to add {{Ic|squid}} to the {{Ic|1=DAEMONS=()}} section of rc.conf if you want it to start on boot.
+
To start squid on boot use this command:
 +
# systemctl enable squid
  
 
== Content Filtering ==
 
== Content Filtering ==
Line 95: Line 107:
 
Follow [[ClamAV|this link]] to install ClamAV on your system.
 
Follow [[ClamAV|this link]] to install ClamAV on your system.
  
Once ClamAV is installed, install HAVP from AUR. Details on installing an AUR package can be found [http://wiki.archlinux.org/index.php/AUR_User_Guidelines#Installing_Packages_from_the_AUR here], and the HAVP package can be found [http://aur.archlinux.org/packages.php?ID=10417 here].
+
Once ClamAV is installed, install HAVP from AUR. Details on installing an AUR package can be found [[AUR_User_Guidelines#Installing_Packages_from_the_AUR|here]], and the HAVP package can be found [https://aur.archlinux.org/packages.php?ID=10417 here].
  
 
=== Configuration ===
 
=== Configuration ===
  
 
Once HAVP is installed, create a user group for the HAVP instance:
 
Once HAVP is installed, create a user group for the HAVP instance:
<pre>
+
useradd havp
adduser havp
+
 
</pre>
+
 
Change the owner of the antivirus logs and temporary file-testing directories to havp :
 
Change the owner of the antivirus logs and temporary file-testing directories to havp :
<pre>
+
chown -R havp:havp /var/run/havp
chown -R havp:havp /var/run/havp
+
chown -R havp:havp /var/log/havp
chown -R havp:havp /var/log/havp
+
</pre>
+
  
 
Add the mandatory lock option to your filesystem (needed by HAVP) : In your /etc/fstab, modify :
 
Add the mandatory lock option to your filesystem (needed by HAVP) : In your /etc/fstab, modify :
<pre>
+
[...] / ext3 defaults 1 1
[...] / ext3 defaults 1 1
+
</pre>
+
 
to :
 
to :
<pre>
+
[...] / ext3 defaults,mand 1 1
[...] / ext3 defaults,mand 1 1
+
 
</pre>
+
 
Then reload your filesystem :
 
Then reload your filesystem :
<pre>
+
mount -o remount /
mount -o remount /
+
</pre>
+
  
 
Add this info in your /etc/squid/squid.conf :
 
Add this info in your /etc/squid/squid.conf :
<pre>
+
cache_peer 127.0.0.1 parent 8080 0 no-query no-digest no-netdb-exchange default
cache_peer 127.0.0.1 parent 8080 0 no-query no-digest no-netdb-exchange default
+
cache_peer_access 127.0.0.1 allow all
cache_peer_access 127.0.0.1 allow all
+
</pre>
+
  
 
Make sure your port in your /etc/havp/havp.config matches the cache_peer port in /etc/squid/squid.conf.
 
Make sure your port in your /etc/havp/havp.config matches the cache_peer port in /etc/squid/squid.conf.
  
 
=== Testing ===
 
=== Testing ===
Reload your squid and start HAVP :
+
Reload your squid and start HAVP:
<pre>
+
systemctl restart squid
/etc/rc.d/squid restart
+
systemctl start havp
/etc/rc.d/havp start
+
</pre>
+
  
 
Don't forget to add HAVP to your rc.conf if your want it to launch on boot :
 
Don't forget to add HAVP to your rc.conf if your want it to launch on boot :
<pre>
+
systemctl enable squid
DAEMONS=([...] squid havp [...]_
+
systemctl enable havp
</pre>
+
  
 
You can try the antivirus capabilities with a test virus (not a real virus) available [http://www.eicar.org/anti_virus_test_file.htm here].
 
You can try the antivirus capabilities with a test virus (not a real virus) available [http://www.eicar.org/anti_virus_test_file.htm here].
  
 
== Transparent web proxy ==
 
== Transparent web proxy ==
Transparency happens by redirecting all www requests eth0 picks up, to Squid. You'll need to indicate Squid that it is running like a transparent web proxy by adding the {{Ic|transparent}} parameter to the {{Ic|http_port}} option:
+
Transparency happens by redirecting all www requests eth0 picks up, to Squid. You'll need to indicate Squid that it is running like a transparent web proxy by adding the {{Ic|intercept}} (for squid 3.2) parameter to the {{Ic|http_port}} option:
   http_port 3128 '''transparent'''
+
   http_port 3128 '''intercept'''
  
 +
=== iptables ===
 +
From a terminal with root privileges, run:
 +
# gid=`id -g proxy`
 +
# iptables -t nat -A OUTPUT -p tcp --dport 80 -m owner --gid-owner $gid -j ACCEPT
 +
# iptables -t nat -A OUTPUT -p tcp --dport 80 -j DNAT --to-destination SQUIDIP:3128
 +
# iptables-save > /etc/iptables/iptables.rules
 +
 +
Then start Iptables:
 +
# systemctl start iptables.service
 +
 +
Replace SQUIDIP with the public IP(s) which squid may use for its listening port and outbound connections.
 +
 +
{{Note|If you are using a content filtering solution, you should put the port for it, not the Squid port, and you need to remove the {{Ic|intercep}} option in the http_port line.}}
 
=== Shorewall ===
 
=== Shorewall ===
 
Edit /etc/shorewall/rules and add
 
Edit /etc/shorewall/rules and add
Line 153: Line 165:
 
  ACCEPT $FW net tcp www # allow Squid to fetch the www content
 
  ACCEPT $FW net tcp www # allow Squid to fetch the www content
  
  /etc/rc.d/shorewall restart
+
  systemctl restart shorewall
 +
 
 +
== HTTP Authentication ==
 +
 
 +
Squid can be configured to require a user and password in order to use it. We will use [[wikipedia:Digest_access_authentication|digest http auth]]
 +
 
 +
First create a users file with {{Ic|htdigest -c /etc/squid/users MyRealm username}}. Enter a password when prompted.
 +
 
 +
Then add these lines to your {{Ic|squid.conf}}:
 +
 
 +
    auth_param digest program /usr/lib/squid/digest_file_auth -c /etc/squid/users
 +
    auth_param digest children 5
 +
    auth_param digest realm MyRealm
 +
   
 +
    acl users proxy_auth REQUIRED
 +
    http_access allow users
 +
 
 +
And restart squid. Now you will be prompted to enter a username and password when accessing the proxy.
 +
 
 +
You can add more users with {{Ic|htdigest /etc/squid/users MyRealm newuser}}. You probably would like to install Apache package, which contains {{Ic|htdigest}} tool.
 +
 
 +
{{Note|Be aware that {{Ic|http_access}} rules cascade, so you need to set them in the desired order.}}
 +
 
 +
=== NTLM ===
 +
 
 +
{{Warning|NTLM is deprecated and has security problems.}}
 +
 
 +
Set up [[samba]] and winbindd and test it with
 +
  ntlm_auth --username=DOMAIN\\user
 +
 
 +
Grant r-x access to /var/cache/samba/winbindd_privileged/ directory for squid user/group
 +
 
 +
Then add something like this to squid.conf:
 +
 
 +
  auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp
 +
  auth_param ntlm children 5
 +
  auth_param ntlm max_challenge_reuses 0
 +
  auth_param ntlm max_challenge_lifetime 2 minutes
 +
  auth_param ntlm keep_alive off
 +
 
 +
  acl ntlm_users proxy_auth REQUIRED
 +
  http_access allow ntlm_users
 +
  http_access deny all
  
 
== Additional Resources ==
 
== Additional Resources ==
* [http://wiki.gotux.net/config:squid Elite Proxy Config Example] - UnOfficial Site
+
* [http://gotux.net/arch-linux/squid-proxy-server/ Elite Proxy Config Example]

Revision as of 05:19, 28 April 2013

Tango-edit-clear.pngThis article or section needs language, wiki syntax or style improvements.Tango-edit-clear.png

Reason: please use the first argument of the template to provide a brief explanation. (Discuss in Talk:Squid#)

From the squid website:

Squid is a caching proxy for the Web supporting HTTP, HTTPS, FTP, and more. It reduces bandwidth and improves response times by caching and reusing frequently-requested web pages. Squid has extensive access controls and makes a great server accelerator. It runs on Unix and Windows and is licensed under the GNU GPL.

While squid works wonderfully in large corporations and schools, it can also benefit the home user too. However, if you're looking for a more lightweight single-user proxy, you should try Polipo.

Installation

Install squid available in the Official Repositories.

Configuration

By default, the cache directories will be created in /var/cache/squid, and the appropriate permissions set up for those directories. However, for greater control, we need to delve into /etc/squid/squid.conf.

Everything is well commented, but if you want to strip the comments out you should run:

sed -i "/^#/d;/^ *$/d" /etc/squid/squid.conf

The following options might be of some use to you. If you do not have the option present in your configuration file, add it!

  • http_port - Sets the port that Squid binds to on your local machine. You can have Squid bind to multiple ports by specifying multiple http_port lines. By default, Squid binds to port 3128.
http_port 3128
http_port 3129
  • http_access - This is an access control list for who is allowed to use the proxy. By default only localhost is allowed to access the proxy. For testing purposes, you may want to change the option http_access deny all to http_access allow all, which will allow anyone to connect to your proxy. If you wanted to just allow access to your subnet, you can do:
acl ip_acl src 192.168.1.0/24
http_access allow ip_acl
http_access deny all
  • cache_mgr - This is the email address of the cache manager.
cache_mgr squid.admin@example.com
  • shutdown_lifetime - Specifies how long Squid should wait when its service is asked to stop. If you're running squid on your desktop PC, you may want to set this to something short.
shutdown_lifetime 10 seconds
  • cache_mem - This is how much memory you want Squid to use to keep objects in memory rather than writing them to disk. Squid's total memory usage will exceed this! By default this is 8MB, so you might want to increase it if you have lots of RAM available.
cache_mem 64 MB
  • visible_hostname - hostname that will be shown in status/error messages
visible_hostname cerberus
  • cache_peer - If you want your Squid to go through another proxy server, rather than directly out to the Internet, you need to specify it here.
  • login - Use this option if the parent proxy requires authentication.
  • never_direct - Tells the cache to never go direct to the internet to retrieve a page. You will want this if you have set the option above.
cache_peer 10.1.1.100 parent 8080 0 no-query default login=user:password
never_direct allow all
  • maximum_object_size - The largest size of a cached object. By default this is small (256KB I think), so if you have a lot of disk space you will want to increase the size of it to something reasonable.
maximum_object_size 10 MB
Note: After defining a new cache_dir it maybe necessary to initialize the caches directory structure with this command: squid -zN -z for Create missing swap directories and -N for No daemon mode.
  • cache_dir - This is your cache directory, where all the cached files are stored. There are many options here, but the format should generally go like:
cache_dir <storage type> <directory> <size in MB> 16 256

So, in the case of a school's internet proxy:

cache_dir diskd /cache0 200000 16 256

If you change the cache directory from defaults, you must set the correct permissions on the cache directory before starting Squid, else it won't be able to create its cache directories and will fail to start.

Accessing services on local hostnames

If you plan to access web servers on the LAN using hostnames that are not fully-defined (e.g. http://mywebapp), you may need to enable the dns_defnames option. Without this option, Squid will make a DNS request for the hostname verbatim (mywebapp), which may fail, depending on your LAN's DNS setup. With the option enabled, Squid will append any domain configured in /etc/resolv.conf when making the request (e.g. mywebapp.company.local).

dns_defnames on

Starting

Once you have finished your configuration, you should check that your configuration file is correct:

# squid -k check

Then create your cache directories:

# squid -z

Then you can start Squid!

# systemctl start squid

To start squid on boot use this command:

# systemctl enable squid

Content Filtering

If you're looking for a content filtering solution to work with Squid, you should check out the very powerful DansGuardian.

Frontend

If you'd like a web-based frontend for managing Squid, Webmin is your best bet.

Ad blocking with adzapper

Adzapper is a plugin for Squid. It catches ads of all sorts (even Flash animations) and replaces them with an image of your choice, so the layout of the page isn't altered very much.

Installation

Adzapper is no longer in the community repository, but it can be found in the AUR.

Configuration

echo "redirect_program /usr/bin/adzapper.wrapper" >> /etc/squid/squid.conf

(squid 2.6.STABLE13-1)

echo "url_rewrite_program /usr/bin/adzapper.wrapper" >> /etc/squid/squid.conf
echo "url_rewrite_children 10" >> /etc/squid/squid.conf

If you want, you can configure adzapper to your liking. The configuration out of the box works wonderfully well though.

nano /etc/adzapper/adzapper.conf

Anti-virus layer

Adding Anti-virus capabilities to Squid is done using the HAVP program to interface it with ClamAV.

Installing dependencies

Follow this link to install ClamAV on your system.

Once ClamAV is installed, install HAVP from AUR. Details on installing an AUR package can be found here, and the HAVP package can be found here.

Configuration

Once HAVP is installed, create a user group for the HAVP instance:

useradd havp

Change the owner of the antivirus logs and temporary file-testing directories to havp :

chown -R havp:havp /var/run/havp
chown -R havp:havp /var/log/havp

Add the mandatory lock option to your filesystem (needed by HAVP) : In your /etc/fstab, modify :

[...] / ext3 defaults 1 1

to :

[...] / ext3 defaults,mand 1 1

Then reload your filesystem :

mount -o remount /

Add this info in your /etc/squid/squid.conf :

cache_peer 127.0.0.1 parent 8080 0 no-query no-digest no-netdb-exchange default
cache_peer_access 127.0.0.1 allow all

Make sure your port in your /etc/havp/havp.config matches the cache_peer port in /etc/squid/squid.conf.

Testing

Reload your squid and start HAVP:

systemctl restart squid
systemctl start havp

Don't forget to add HAVP to your rc.conf if your want it to launch on boot :

systemctl enable squid
systemctl enable havp

You can try the antivirus capabilities with a test virus (not a real virus) available here.

Transparent web proxy

Transparency happens by redirecting all www requests eth0 picks up, to Squid. You'll need to indicate Squid that it is running like a transparent web proxy by adding the intercept (for squid 3.2) parameter to the http_port option:

 http_port 3128 intercept

iptables

From a terminal with root privileges, run:

# gid=`id -g proxy`
# iptables -t nat -A OUTPUT -p tcp --dport 80 -m owner --gid-owner $gid -j ACCEPT
# iptables -t nat -A OUTPUT -p tcp --dport 80 -j DNAT --to-destination SQUIDIP:3128
# iptables-save > /etc/iptables/iptables.rules

Then start Iptables:

# systemctl start iptables.service

Replace SQUIDIP with the public IP(s) which squid may use for its listening port and outbound connections.

Note: If you are using a content filtering solution, you should put the port for it, not the Squid port, and you need to remove the intercep option in the http_port line.

Shorewall

Edit /etc/shorewall/rules and add

REDIRECT	loc	3128	tcp	www # redirect to Squid on port 3128
ACCEPT		$FW	net	tcp	www # allow Squid to fetch the www content
systemctl restart shorewall

HTTP Authentication

Squid can be configured to require a user and password in order to use it. We will use digest http auth

First create a users file with htdigest -c /etc/squid/users MyRealm username. Enter a password when prompted.

Then add these lines to your squid.conf:

   auth_param digest program /usr/lib/squid/digest_file_auth -c /etc/squid/users
   auth_param digest children 5
   auth_param digest realm MyRealm
   
   acl users proxy_auth REQUIRED
   http_access allow users

And restart squid. Now you will be prompted to enter a username and password when accessing the proxy.

You can add more users with htdigest /etc/squid/users MyRealm newuser. You probably would like to install Apache package, which contains htdigest tool.

Note: Be aware that http_access rules cascade, so you need to set them in the desired order.

NTLM

Warning: NTLM is deprecated and has security problems.

Set up samba and winbindd and test it with

 ntlm_auth --username=DOMAIN\\user

Grant r-x access to /var/cache/samba/winbindd_privileged/ directory for squid user/group

Then add something like this to squid.conf:

 auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp
 auth_param ntlm children 5
 auth_param ntlm max_challenge_reuses 0
 auth_param ntlm max_challenge_lifetime 2 minutes
 auth_param ntlm keep_alive off
 acl ntlm_users proxy_auth REQUIRED
 http_access allow ntlm_users
 http_access deny all

Additional Resources