Difference between revisions of "Polipo"

From ArchWiki
Jump to navigation Jump to search
m (typo)
m (rm italic from quote)
 
(80 intermediate revisions by 35 users not shown)
Line 1: Line 1:
[[Category:Networking (English)]]
+
[[Category:Proxy servers]]
[[Category:Daemons and system services (English)]]
+
[[ja:Polipo]]
{{expansion}}
+
[[zh-hans:Polipo]]
 +
{{Warning|Polipo is no longer maintained, It's recommended to use another solution ([[Squid]], [[Privoxy]]...) instead.}}
 +
 
 
From [http://www.pps.jussieu.fr/~jch/software/polipo/ Polipo's site]:
 
From [http://www.pps.jussieu.fr/~jch/software/polipo/ Polipo's site]:
:"''Polipo is a small and fast caching web proxy (a web cache, an HTTP proxy, a proxy server). While Polipo was designed to be used by one person or a small group of people, there is nothing that prevents it from being used by a larger group.''"
 
  
Unlike [[Squid]], Polipo is very light on resources and simple to configure. This makes it ideal for single user systems and other uncomplicated setups. Do keep in mind; however, that this versatility comes at a cost; Polipo will increase its space usage without restriction as it is not aware of how big its disk cache grows. This perceived fault is by design, since omitting these sanity checks drastically reduces Polipo's memory usage and overall toll on the system. A practical way of restricting disk usage is by making Polipo run as its own user and employing [[disk quota]].
+
:Polipo is a small and fast caching web proxy (a web cache, an HTTP proxy, a proxy server). While Polipo was designed to be used by one person or a small group of people, there is nothing that prevents it from being used by a larger group.
 +
 
 +
Unlike [[Squid]], Polipo is very light on resources and simple to configure. This makes it ideal for single user systems and other uncomplicated setups. Do keep in mind, however, that this versatility comes at a cost: Polipo will increase its space usage without restriction as it is not aware of how big its disk cache grows. This perceived fault is by design, since omitting these sanity checks drastically reduces Polipo's memory usage and overall toll on the system. A practical way of restricting disk usage is by making Polipo run as its own user and employing [[disk quota]].
  
 
The following covers installing and setting up Polipo.
 
The following covers installing and setting up Polipo.
  
==Installation==
+
== Installation ==
Polipo is currently available in the [[AUR]]. Using an [[AUR helper]] like [[yaourt]], install by entering:
 
$ yaourt -S polipo
 
or install the newer development version instead:
 
$ yaourt -S polipo-git
 
  
==Improving Polipo==
+
[[Install]] the {{Pkg|polipo}} package.
{{note|the git version already has these improvements, except for the designated Polipo user modification.}}
 
The current Polipo package is missing a set of features users might find desirable, namely: proper [[daemon]] behavior, including placing files in {{filename|/var/run}} and a call to cleanse Polipo's cache; a cronjob that routinely performs the latter; and finally, a restricted "polipo" user to address security and maintainability concerns.
 
  
To partially fix these issues, replace the daemon script with the following:
+
== Starting the daemon ==
{{file|name=/etc/rc.d/polipo|content=
 
<nowiki>
 
#!/bin/bash
 
. /etc/rc.conf
 
. /etc/rc.d/functions
 
  
DAEMON=polipo
+
To start the polipo daemon:
ARGS="daemonise=true pidFile=/var/run/$DAEMON/$DAEMON.pid"
 
PID=$(pidof -o %PPID /usr/bin/$DAEMON)
 
  
case $1 in
+
# systemctl start polipo
    start)
 
        stat_busy "Starting $DAEMON"
 
        if ck_daemon $DAEMON; then
 
            [[ ! -d /var/run/$DAEMON ]] && install -d $DAEMON /var/run/$DAEMON
 
            /usr/bin/$DAEMON $ARGS >/dev/null 2>&1
 
            if [[ $? != 0 ]]; then
 
                stat_fail
 
            else
 
                add_daemon polipo
 
                stat_done
 
            fi
 
        else
 
            stat_fail
 
        fi
 
    ;;
 
    stop)
 
        stat_busy "Stopping $DAEMON"
 
        if ! ck_daemon $DAEMON; then
 
            kill $PID >/dev/null 2>&1
 
            if [[ $? != 0 ]]; then
 
                stat_fail
 
            else
 
                rm_daemon $DAEMON
 
                stat_done
 
            fi
 
        else
 
            stat_fail
 
        fi
 
    ;;
 
    purge)
 
        stat_busy "Purging $DAEMON"
 
        [[ ! -d /var/run/$DAEMON ]] && install -d $DAEMON /var/run/$DAEMON
 
        if ! ck_daemon $DAEMON; then
 
            kill -USR1 $PID >/dev/null 2>&1
 
            sleep 1
 
            /usr/bin/$DAEMON -x $ARGS >/dev/null 2>&1 || stat_fail
 
            kill -USR2 $PID >/dev/null 2>&1
 
            stat_done
 
        else
 
            /usr/bin/$DAEMON -x $ARGS >/dev/null 2>&1 || stat_fail
 
            stat_done
 
        fi
 
    ;;
 
    restart)
 
        $0 stop
 
        $0 start
 
    ;;
 
    *)
 
        echo "usage: $0 {start|stop|restart|purge}"
 
    ;;
 
esac
 
</nowiki>
 
}}
 
  
And save the cron file in {{filename|/etc/cron.weekly/polipo}}:
+
To start it automatically at boot:
#!/bin/sh
 
/etc/rc.d/polipo purge >/dev/null 2>&1
 
  
Make it executable:
+
  # systemctl enable polipo
  # chmod +x /etc/cron.weekly/polipo
 
  
===Run Polipo as designated user===
+
=== Multiple instances ===
{{note|to-do.}}
 
  
==Starting the daemon==
+
Polipo can also run without super user privileges. To do so, first copy {{ic|/etc/polipo/config.sample}} to a suitable directory:
To start the Polipo daemon:
 
# /etc/rc.d/polipo start
 
  
Add it to {{filename|/etc/rc.conf}} to start it automatically at boot:
+
$ cp /etc/polipo/config.sample ~/.poliporc
DAEMONS=(syslog-ng network netfs '''polipo''' crond)
 
  
===Multiple instances===
+
Edit it so that it points at a writable location, instead of {{ic|/var/cache/polipo}}:
Polipo can also run without super user priveleges. To do so, first copy {{filename|/etc/polipo/config.sample}} to a suitable directory:
 
$ cp /etc/polipo/config.sample ~/.poliporc
 
  
Edit it so that it points at a writable location, instead of {{filename|/var/cache/polipo}}:
 
 
  # Uncomment this if you want to put the on-disk cache in a
 
  # Uncomment this if you want to put the on-disk cache in a
 
  # non-standard location:
 
  # non-standard location:
Line 112: Line 39:
  
 
Create the cache directory:
 
Create the cache directory:
 +
 
  $ mkdir ~/.polipo-cache
 
  $ mkdir ~/.polipo-cache
  
 
Finally, launch Polipo with the new configuration:
 
Finally, launch Polipo with the new configuration:
 +
 
  $ polipo -c ~/.poliporc
 
  $ polipo -c ~/.poliporc
  
==Configuration==
+
== Configuration ==
Management is mostly performed in {{filename|/etc/polipo/config}}. Most users can opt for using the sample configuration file, which is sufficient for most situations and well documented.
+
 
 +
Management is mostly performed in {{ic|/etc/polipo/config}}. Most users can opt for using the sample configuration file, which is sufficient for most situations and well documented.
 +
 
 
  # cd /etc/polipo; cp config.sample config
 
  # cd /etc/polipo; cp config.sample config
 +
 +
One element of configuration that warrants mentioning is polipo's default behavior of blocking outbound connections by port. There are two variables in polipo's config file that control allowed outbound ports. {{Ic|allowedPorts}} specifies ports for outbound HTTP connections. It defaults to 80-100 and 1024-65535. {{Ic|tunnelAllowedPorts}} specifies ports polipo will allow tunnel traffic to as well as HTTPS traffic. By default it is much more restricted: "''It defaults to allowing ssh, HTTP, https, rsync, IMAP, imaps, POP, pops, Jabber, CVS and Git traffic.''"
 +
 +
If you see a "403 Forbidden Port" error message from polipo when attempting to browse to a host:port, you need to configure polipo to accept traffic to more ports for either HTTP or HTTPS. To set them wide open, add the following to {{ic|/etc/polipo/config}}:
 +
 +
allowedPorts = 1-65535
 +
tunnelAllowedPorts = 1-65535
  
 
Unlike other proxies, Polipo needs to be restarted after alterations.
 
Unlike other proxies, Polipo needs to be restarted after alterations.
  
===Browser===
+
=== Browser ===
Set the browser so that it uses {{codeline|localhost:8123}} for proxying. Be sure to disable the browser's disk cache to avoid redundant IO operations and bad performance.
+
 
 +
Set the browser so that it uses {{Ic|localhost:8123}} for proxying. Be sure to disable the browser's disk cache to avoid redundant IO operations and bad performance. If {{Ic|localhost:8123}} does not work, try matching {{Ic|/etc/hosts}} like {{Ic|''hostname''.localdomain:8123}}.
 +
 
 +
=== Tunneling ===
 +
 
 +
{{note|According to the [http://www.pps.jussieu.fr/~jch/software/polipo/faq.html Polipo FAQ] on "intercepting proxy" this is not possible/supported!}}
  
===Tunneling===
 
 
{{note|this requires to run Polipo as its own user.}}
 
{{note|this requires to run Polipo as its own user.}}
 +
 
Instead of manually configuring each browser or other utilities that might benefit from Polipo's caching, one can also use [[iptables]] to route traffic through polipo.
 
Instead of manually configuring each browser or other utilities that might benefit from Polipo's caching, one can also use [[iptables]] to route traffic through polipo.
  
After installing iptables, add the appropiate rules to {{filename|/etc/iptables/iptables.rules}}:
+
After installing iptables, add the appropiate rules to {{ic|/etc/iptables/iptables.rules}}:
 +
 
 
  *nat
 
  *nat
 
  :PREROUTING ACCEPT [0:0]
 
  :PREROUTING ACCEPT [0:0]
 
  :POSTROUTING ACCEPT [0:0]
 
  :POSTROUTING ACCEPT [0:0]
 
  :OUTPUT ACCEPT [0:0]
 
  :OUTPUT ACCEPT [0:0]
  ''-A OUTPUT -p tcp --dport 80 -m owner --uid-owner polipo -j ACCEPT''
+
  ''-A OUTPUT -p tcp --dport 80 -m owner ! --uid-owner polipo -j ACCEPT''
 
  ''-A OUTPUT -p tcp --dport 80 -j REDIRECT --to-ports 8123''
 
  ''-A OUTPUT -p tcp --dport 80 -j REDIRECT --to-ports 8123''
 
  COMMIT
 
  COMMIT
 +
 
This routes HTTP traffic through Polipo. Remove all proxy settings from browsers, if any, and restart iptables.
 
This routes HTTP traffic through Polipo. Remove all proxy settings from browsers, if any, and restart iptables.
  
===Privoxy===
+
=== Privoxy ===
 +
 
 
[[Privoxy]] is a proxy useful for intercepting advertisement and other undesirables.
 
[[Privoxy]] is a proxy useful for intercepting advertisement and other undesirables.
  
Line 146: Line 92:
  
 
In other words:
 
In other words:
*point the browser at Privoxy: {{codeline|localhost:8118}}
 
*and direct Privoxy traffic to Polipo: {{codeline|forward / localhost:8123}} in the Privoxy configuration file.
 
  
===Tor===
+
* point the browser at Privoxy: {{Ic|localhost:8118}}
 +
 
 +
* and direct Privoxy traffic to Polipo: {{Ic|forward / localhost:8123}} in the Privoxy configuration file.
 +
 
 +
=== Tor ===
 +
 
 +
{{Warning|The Tor Project advises against transparently routing traffic through Tor [https://www.torproject.org/docs/faq.html.en#TBBOtherBrowser] [https://www.torproject.org/docs/faq.html.en#TBBSocksPort], and strongly recommends using only the Tor Browser [https://www.torproject.org/download/download.html.en#warning]. Consider instead using an [https://trac.torproject.org/projects/tor/wiki/doc/TorifyHOWTO/IsolatingProxy Isolating Proxy].}}
 +
 
 
[[Tor]] is an anonymizing proxy network.
 
[[Tor]] is an anonymizing proxy network.
  
To use Polipo with Tor, uncomment or include the following in {{codeline|/etc/polipo/config}}:
+
To use Polipo with Tor, uncomment or include the following in {{Ic|/etc/polipo/config}}:
 +
 
 
  socksParentProxy = localhost:9050
 
  socksParentProxy = localhost:9050
 +
socksProxyType = socks5
 +
 +
=== DansGuardian ===
 +
 +
[[DansGuardian]] is a web content filter. The only difference to using [[DansGuardian]] with Polipo (rather than squid or tinyproxy) is that in {{ic|dansguardian.conf}} the proxyport needs to be set to polipo's 8123:
 +
 +
# the port DansGuardian connects to proxy on
 +
proxyport = 8123
 +
 +
== Troubleshooting ==
 +
 +
=== DNS Error ===
 +
 +
If the network is started in background there could be a error like this in the Polipo log:
 +
 +
Couldn't send DNS query: Connection refused
 +
Falling back on gethostbyname.
 +
Getaddrinfo failed: Temporary name server failure
 +
Host ***.com lookup failed: Getaddrinfo failed: Temporary name server failure (131072).
 +
 +
This error occurs because in background mode the network hasn't initialised before Polipo wants to connect to the DNS server (especially using DHCP). Solving this error is possible on three ways:
 +
 +
* Do not start the net-profiles in background mode (probably not wanted).
 +
* Set {{ic|dnsNameServer}} manually on the wanted DNS server.
 +
* Or add {{ic|sleep 10}} (or more, it depends) near the beginning of the Polipo daemon script {{ic|/etc/rc.d/polipo}} in the start section. This will make Polipo start after the network has initialised.
 +
 +
See [https://bbs.archlinux.org/viewtopic.php?id=86452 this thread] for more information on this topic.
 +
 +
== See also ==
  
==More resources==
 
 
*[http://www.pps.jussieu.fr/~jch/software/polipo/faq.html Polipo FAQ]
 
*[http://www.pps.jussieu.fr/~jch/software/polipo/faq.html Polipo FAQ]
 
*[http://www.pps.jussieu.fr/~jch/software/polipo/manual/index.html The Polipo Manual]
 
*[http://www.pps.jussieu.fr/~jch/software/polipo/manual/index.html The Polipo Manual]

Latest revision as of 13:20, 24 October 2018

Warning: Polipo is no longer maintained, It's recommended to use another solution (Squid, Privoxy...) instead.

From Polipo's site:

Polipo is a small and fast caching web proxy (a web cache, an HTTP proxy, a proxy server). While Polipo was designed to be used by one person or a small group of people, there is nothing that prevents it from being used by a larger group.

Unlike Squid, Polipo is very light on resources and simple to configure. This makes it ideal for single user systems and other uncomplicated setups. Do keep in mind, however, that this versatility comes at a cost: Polipo will increase its space usage without restriction as it is not aware of how big its disk cache grows. This perceived fault is by design, since omitting these sanity checks drastically reduces Polipo's memory usage and overall toll on the system. A practical way of restricting disk usage is by making Polipo run as its own user and employing disk quota.

The following covers installing and setting up Polipo.

Installation

Install the polipo package.

Starting the daemon

To start the polipo daemon:

# systemctl start polipo

To start it automatically at boot:

# systemctl enable polipo

Multiple instances

Polipo can also run without super user privileges. To do so, first copy /etc/polipo/config.sample to a suitable directory:

$ cp /etc/polipo/config.sample ~/.poliporc

Edit it so that it points at a writable location, instead of /var/cache/polipo:

# Uncomment this if you want to put the on-disk cache in a
# non-standard location:
diskCacheRoot = "~/.polipo-cache/"

Create the cache directory:

$ mkdir ~/.polipo-cache

Finally, launch Polipo with the new configuration:

$ polipo -c ~/.poliporc

Configuration

Management is mostly performed in /etc/polipo/config. Most users can opt for using the sample configuration file, which is sufficient for most situations and well documented.

# cd /etc/polipo; cp config.sample config

One element of configuration that warrants mentioning is polipo's default behavior of blocking outbound connections by port. There are two variables in polipo's config file that control allowed outbound ports. allowedPorts specifies ports for outbound HTTP connections. It defaults to 80-100 and 1024-65535. tunnelAllowedPorts specifies ports polipo will allow tunnel traffic to as well as HTTPS traffic. By default it is much more restricted: "It defaults to allowing ssh, HTTP, https, rsync, IMAP, imaps, POP, pops, Jabber, CVS and Git traffic."

If you see a "403 Forbidden Port" error message from polipo when attempting to browse to a host:port, you need to configure polipo to accept traffic to more ports for either HTTP or HTTPS. To set them wide open, add the following to /etc/polipo/config:

allowedPorts = 1-65535
tunnelAllowedPorts = 1-65535

Unlike other proxies, Polipo needs to be restarted after alterations.

Browser

Set the browser so that it uses localhost:8123 for proxying. Be sure to disable the browser's disk cache to avoid redundant IO operations and bad performance. If localhost:8123 does not work, try matching /etc/hosts like hostname.localdomain:8123.

Tunneling

Note: According to the Polipo FAQ on "intercepting proxy" this is not possible/supported!
Note: this requires to run Polipo as its own user.

Instead of manually configuring each browser or other utilities that might benefit from Polipo's caching, one can also use iptables to route traffic through polipo.

After installing iptables, add the appropiate rules to /etc/iptables/iptables.rules:

*nat
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A OUTPUT -p tcp --dport 80 -m owner ! --uid-owner polipo -j ACCEPT
-A OUTPUT -p tcp --dport 80 -j REDIRECT --to-ports 8123
COMMIT

This routes HTTP traffic through Polipo. Remove all proxy settings from browsers, if any, and restart iptables.

Privoxy

Privoxy is a proxy useful for intercepting advertisement and other undesirables.

According to Polipo's developer, in order to get the privacy enhancements of Privoxy and much (but not all) of the performance of Polipo, one should place Polipo upstream of Privoxy.

In other words:

  • point the browser at Privoxy: localhost:8118
  • and direct Privoxy traffic to Polipo: forward / localhost:8123 in the Privoxy configuration file.

Tor

Warning: The Tor Project advises against transparently routing traffic through Tor [1] [2], and strongly recommends using only the Tor Browser [3]. Consider instead using an Isolating Proxy.

Tor is an anonymizing proxy network.

To use Polipo with Tor, uncomment or include the following in /etc/polipo/config:

socksParentProxy = localhost:9050
socksProxyType = socks5

DansGuardian

DansGuardian is a web content filter. The only difference to using DansGuardian with Polipo (rather than squid or tinyproxy) is that in dansguardian.conf the proxyport needs to be set to polipo's 8123:

# the port DansGuardian connects to proxy on
proxyport = 8123

Troubleshooting

DNS Error

If the network is started in background there could be a error like this in the Polipo log:

Couldn't send DNS query: Connection refused
Falling back on gethostbyname.
Getaddrinfo failed: Temporary name server failure
Host ***.com lookup failed: Getaddrinfo failed: Temporary name server failure (131072).

This error occurs because in background mode the network hasn't initialised before Polipo wants to connect to the DNS server (especially using DHCP). Solving this error is possible on three ways:

  • Do not start the net-profiles in background mode (probably not wanted).
  • Set dnsNameServer manually on the wanted DNS server.
  • Or add sleep 10 (or more, it depends) near the beginning of the Polipo daemon script /etc/rc.d/polipo in the start section. This will make Polipo start after the network has initialised.

See this thread for more information on this topic.

See also