High Performance Firewall (Español)

From ArchWiki
Revision as of 17:18, 12 October 2007 by CarLost (Talk | contribs) (New page: Category: Networking (Español) = Firewall/nat de alto rendimiento usando iptables, VLANs e ipoute2 = =Introducción= Holas, este es mi segundo Wiki, el primero es el mismo version ...)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Firewall/nat de alto rendimiento usando iptables, VLANs e ipoute2


Holas, este es mi segundo Wiki, el primero es el mismo version Ingles, espero les sirva.

Imagina esto, tienes mas de 2 redes separadas por protocolo de VLAN trunk (IEEE 802.1q), las cuales llegan a ti via un punto de red Tagging (troncal) a través de un Switch administrable, en una linea 10/100/1000 (lo mejor es 100 FD naturalmente)

Tienes que compartir internet a un número realmente grande de equipos y mantener un buen rendimiento. La primera opción es separar las redes en un número igual de puertas "access" (no TAG) y tal ves un numero igual o mayor de equipos firewalls. Esto no es realmente barato, pero funciona.

La segunda opción es la que hice. La historua de como comenxo todo esto esta relacionada con una emergencia donde se frieron un grupo de Cisco PIXs. No voy a profundizar en el tema.

Los Echos

Tengo alrededor de 4 redes con mascara 21, esto es 8 clases C cada una!!!. Esto genera una cantidad enorme de Direcciones MACs y ahun más peligroso un número ENORME de BRADCAST, Esto es una locura, pero asi es como trabaja mi empresa.
Me diero 30 IPs públicas en 3 grupos
Y un PC, uno pequeño, no era la gran cosa.... Solo para probar. Después (8 meses) lo cambiamos por un PC real. Un monstruo!!!
Más tarde le añadimos una nueva clase C con un número creciente de subredes ruteadas .... EXTRA

El Trabajo

Soporte de VLAN

La primera cosa que tenemos que hacer es darle al kernel la capacidad de trabajar con JumboFrames (paquetes con TAG). Esto se hace añadiendo el modulo 8021q al kernel

# modprobe 8021q

Y/o ponerlo en la sección modules en /etc/rc.conf

Lo siguiente es crear las tarjetas virtiales con el comando vconfig. Supongamos que tenemos las VLANes 20, 30, 40 y 50 en nuestra red troncal

# vconfig add ethX 20
# vconfig add ethX 30
# vconfig add ethX 50

Donde ethX es la nuestra tarjeta troncal. Para ver las tarjetas creadas colocamos el comando 'ifconfig -a' y asi obtenemos una lista

So .... next just put the private address as you want. For example (using eth1 like a troncal)

# ifconfig eth1.20 netmask
# ifconfig eth1.30 netmask

I will not explain the number of host neither the mask.... I do a kind of daemon with this, a vlan.conf file in /etc and a vland in /etc/rc.d... I can share if you want

The Firewall

It is really easy make a firewall/nat with iptables there is a lot of information around there...

Just take in mind that you will work with a lot of traffic, this mean, a lot of CPU usage, so, keep your rules tiny, just the necessary. Accept all by default just want to make NAT, no more.... Maybe is good accelerate some ports, (80,443,25,110,21,20,53 etc) remember, every packet running trough our firewall will past trough every rule until it match to one or fall in the pool (default policy).

The round robin NAT

Let's suppose we have a one ip: 200.aaa.bbb.6 and our gateway is 200.aaa.bbb.1. we can safely put this parameters by default in our configuration. It will not get participation at all in our firewall.

I say I have 3 groups of 10 IPs each to play...... we'll define the NEXT in our firewall script:


And the next important line is:

iptables -t nat -A POSTROUTING -s  -j SNAT --to $Gr1 #ACCESS VLAN 10
iptables -t nat -A POSTROUTING -s  -j SNAT --to $Gr2 #ACCESS VLAN 20
iptables -t nat -A POSTROUTING -s  -j SNAT --to $Gr1 #ACCESS VLAN 30
.... etc

You can repeat the groups for access, subdivide the networks ETC, iptables make a round robin over the Gr1, Gr2 and Gr3 by default, no modification is needed.

It's not necessary create a virtual card (alias) to every IP in the group.

It's important that every real router know every group and publish its via BGP (or similar) to the neighbors.


To accelerate some ports you can put this in the top of FORWARD chain

iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -p icmp -o eth0 -j ACCEPT
iptables -A FORWARD -p tcp -m multiport --dports 80,443,110,53 -j ACCEPT  # FAST FAST FAST 
iptables -A FORWARD -p udp  --dport 53 -j ACCEPT

This mean:

  • the packets incoming will pass only 1 rule if it is a establish connection
  • the packet incoming will pass 2 rules if is a ping o similar
  • the packet will pass 3 rules if is http, mail or similar
  • and the DNS recuest will pass 3 o 4 rules until go out

The virus outcoming will KILL our machine, and we not need to share "windows" conversatios so, kill them!!!!

iptables -A FORWARD -p tcp --dport 135:139 -j DROP
iptables -A FORWARD -p tcp --dport 445 -j DROP
iptables -A FORWARD -p udp --dport 135:139 -j DROP
iptables -A FORWARD -p udp --dport 445 -j DROP

If you can, before they reach our machine.

The High Performance

We get to the real important part of this howto.

In our run to get a really big numbers of host running trough our machine we miss some things

  1. We forget that is just one NICs to potentially more than 8000 Mac Addresses. The card shared memory is not prepare to this!!!!!
  2. By default iptables is not prepare to make this numbers of connections simultaneously !!!!!!


To the first isue... I get some error messages in the logs realtive to this.... I'm really sorry, I lost this logs and don't remember what they say. But the answer is this... increase the treshold memory to the neighbors. Type this and read:

# cat /proc/sys/net/ipv4/neigh/default/gc_thresh1 
# cat /proc/sys/net/ipv4/neigh/default/gc_thresh2 
# cat /proc/sys/net/ipv4/neigh/default/gc_thresh3 

Next you can put this in the /etc/sysctrl

net.ipv4.neigh.default.gc_thresh1 = 512
net.ipv4.neigh.default.gc_thresh2 = 1024
net.ipv4.neigh.default.gc_thresh3 = 2048

and make sysctl -p to increase to the double!!! (no reboot needed) with this I get no errors!!!!!

The next will need some comprehension about buckets and conntracks and hashsize (the way how iptables manage the nat connections). There is a very good document about this in here. Read it!!!! Some thing are change since IPtables is know as Netfiler.

In resume!!! Put this in your modules section:

MODULES=(8021q 'nf_conntrack hashsize=1048576' nf_conntrack_ftp 
                               ...and other nf_stuff .......)

The last ones is just to avoid some problems tha we have with ftp connections (I thing this is no necessary anymore). The 'nf_conntrack hashsize=1048576' increase the numbers of the hashsize (increase the kernel memory designated to NAT connections) (need reboot or reload module :-) see with dmesg | grep conntrack)

And the next is put some similar to the /etc/sysctrl.conf file

net.netfilter.nf_conntrack_max = 1048576

And do the sysctl -p command

In my case is the same number, thats mean that I have 1 coneccion for bucket!!!! I don't need more!!!! by default NetFilter put rate of 1:8. I.E. 8 conections per bucket!! (I think, not remmember well)..

In our case we get about 600.000 simultaneous connections in 2 1Giga NICs cards, You can see this with the next command

# cat /proc/sys/net/netfilter/nf_conntrack_count

And put this in a snmpd agent to get and graph it in a MRTG/cacti server ..... uuuuuuu homework

A sample here. (Not actualized, not ever here, but I'll try.)


The iproute2

We have 3 big access to Internet!!! this is because we manage 3 class C groups of IPs (some restrictions of BGP) in this firewall. So, we have 3 incoming traffics that we can manage, but only one outgoing!!! our default gateway. This can easily full our outgoing quote, so we have to spare it

First we have to put some new tables to /etc/iproute2/rt_tables file

# echo 200 PRO_1 >> /etc/iproute2/rt_tables
# echo 205 PRO_2 >> /etc/iproute2/rt_tables
# echo 210 PRO_3 >> /etc/iproute2/rt_tables

Can be more, can be less, depends on traffic

Second we have to give a default gateway to this tables

# ip route add default via 200.aaa.bbb.2 table PRO_1
# ip route add default via 200.aaa.bbb.3 table PRO_2
# ip route add default via 200.aaa.bbb.4 table PRO_3

It's recommended but not necesary put the local interfaces to each table

# ip route add via table PRO_1
# ip route add via table PRO_1
# ip route add via table PRO_1
same PRO_2, same PRO_3

The last thing is give the order to the incoming packages

# ip rule add from table PRO_1

Again, you can play with the PRO_X and even you can play with the mask and submask For example we want to give only a one class C to outgoing to PRO_3

# ip rule add from table PRO_3

Put this before the <NET>/21

and then TEST IT!!!!

pick a WinPC in one of the private networks and make a tracert to somewhere!!!

Before this, you can browse to a some MYIPsite like www.whatismyip.com and get your "RightNow" address, test it later and get some other address ETC...


Actually we are using a PC with 1G ram , Intel Pentium 4 2.8GHz Dual and 2 NICs 1Gb (one integrated one PCI-E), the top CPU usage don't get the 80% pic...

Out traffics is about 180 MB IN/OUT.

We get some lost in the cards, about 1 packets per second, it's really low compared with the traffic, so we just don't care..

Nevertheless we get a big problem, the MAIL/SPAM outgoing, provoked by VIRUS/MAD_GUYs, can block all the publics IP in a really short time. (in the public Black Lists).

The answer to this may be not really ethic, but we have to do it.

We put a similar machine before this one with a bridge configuration blocking every port 25 traffic and sending to another anti SPAM/VIRUS/MAIL machine.

To do this is necessary give to the bridge the capacity of VLANs and one virtual card (in bridge mode i.e.'br0.30') in every net to answer to the sender...

It works and the costumer don't note the difference .... Just if you do a telnet anywhere.net 25 you get a Hi I'm a scanner hello answer!!!.. It's really good!!! and works, the mail outgoing is sender from another IP, so we get our 3 groups safe!!!.

Thats all folks, I hope this can help anyone, and it's the only thing I can do to give you something back..

Be strong, be free, 4 ever

CarLost 11:51, 12 October 2007 (EDT) from Chile

PS: English is not my natural language, I ask you forgive me if there is some uncomprehensive stuff, and feel free to improve this if you want!!!!