Difference between revisions of "High Performance Firewall"

From ArchWiki
Jump to: navigation, search
(→‎VLAN support)
(48 intermediate revisions by 12 users not shown)
Line 1: Line 1:
−
== Hight Performance Firewall/Nat with iptables and VLANs and iproute2 ==
+
[[es:High Performance Firewall]]
 +
[[it:High Performance Firewall]]
 +
[[Category:Networking]]
  
−
=Introduction=
+
{{merge|Router}}
 +
{{poor writing}}
  
−
Hi, this is my first intent of making a wiki document, so be warned, this may have some errors or misunderstandings.
+
{{box BLUE||''High Performance Firewall / Nat with iptables and VLANs and iproute2'' }}
  
−
Imagine this, you have more than two networks separated by Virtual Lans protocols (IEEE 801.1q) or VLANs, carried to you by a intelligent/manageable switch on one troncal line 10/100/1000 MB HD/FD (naturally the best is 1000 MB FD).
+
Imagine this, you have more than two networks separated by Virtual Lans protocols (IEEE 802.1q) or VLANs, carried to you by an intelligent/manageable switch on one troncal line 10/100/1000 MB HD/FD (naturally the best is 1000 MB FD).
  
−
You have to share internet to a really BIG numbers of hosts, and maintain a good performance. The firs choice is separate the networks into a equal numbers of ports and maybe a more numbers of firewalls machines. This is not really cost effective, but works.
+
You have to share internet to a really BIG numbers of hosts, and maintain a good performance. The first choice is to separate the networks into a equal numbers of ports and maybe a more numbers of firewalls machines. This is not really cost effective, but works.
  
−
The second one is what I did. the history of how this begin is related to a some emergency/burn/crash/out of a group of Cisco PIXs. I will don't deep in that.
+
The second one is what I did. The history of how this begin is related to a some emergency/burn/crash/out of a group of Cisco PIXs. I won't go too deeply into that.
  
−
===The facts:===
+
===VLAN support===
−
: I have about 4 networks mask 21 i.e a 8 class C each!!! This is a lots of MACs addresses and even more dangerous a really lots of BROADCAST. This is insane but this is the way that my company works.
 
  
−
: They gave me 30 publics IPs addresses in 3 groups.
+
The first thing we have to do is give the kernel the capacity to work with Jumbo Frames. This is done by adding the 8021q module to the kernel
−
 
 
−
: And a machine, a little one, not big deal.... just for testing. Later (about 8 monts) we change the PC. We put a real monster.
 
−
 
 
−
: Later we add a class C network with a routed lot of increasing subnets.... PLUS!!!!
 
−
 
 
−
=The work=
 
−
==VLAN support==
 
−
 
 
−
The first thing we have to do is give to the kernel the capacity of work with Jumbo Frames. This is done by adding the 8021q module to the kernel
 
 
  # modprobe 8021q
 
  # modprobe 8021q
−
and/or put in modules in /etc/rc.conf
+
and/or put in modules in /etc/rc.conf; alternatively using systemd modules are managed in /etc/modules-load.d/
  
 
Next we have to create the virtuals NICs with this command ''vconfig''. Let's suppose we have vlans 20,30,40 and 50 working in our core network.
 
Next we have to create the virtuals NICs with this command ''vconfig''. Let's suppose we have vlans 20,30,40 and 50 working in our core network.
 +
{{note|vconfig deprecated, use iproute2 instead}}
 
  # vconfig add ethX 20
 
  # vconfig add ethX 20
 
  # vconfig add ethX 30
 
  # vconfig add ethX 30
 
  ...
 
  ...
 
  # vconfig add ethX 50
 
  # vconfig add ethX 50
−
Where ''ethX'' is the troncal NIC
+
Where ''ethX'' is the trunk NIC
−
Now if we want to see the interfaces just put '''ifconfig -a''' and we will get a list.
+
Now, if we want to see the interfaces just put '''ifconfig -a''' and we will get a list.
  
−
So .... next just put the private address as you want. For example (using eth1 like a troncal)
+
So .... next just set the private address as you want. For example (using eth1 like a troncal)
 
  # ifconfig eth1.20 192.168.0.1 netmask 255.255.248.0
 
  # ifconfig eth1.20 192.168.0.1 netmask 255.255.248.0
 
  # ifconfig eth1.30 192.168.8.1 netmask 255.255.248.0
 
  # ifconfig eth1.30 192.168.8.1 netmask 255.255.248.0
Line 41: Line 35:
 
   etc.
 
   etc.
 
I will not explain the number of host neither the mask....
 
I will not explain the number of host neither the mask....
 +
I do a kind of daemon with this, a vlan.conf file in /etc and a vland in /etc/rc.d... I can share this if you want?
  
 +
====The round robin NAT====
  
−
==The Firewall==
+
Let's suppose we have a one ip: 200.aaa.bbb.6 and our gateway is 200.aaa.bbb.1. we can safely put these parameters by default in our configuration. It will not get participation at all in our firewall.
−
 
 
−
It is really easy make a firewall/nat with ''iptables'' there is a lot of information around there...
 
−
 
 
−
Just make in mind that you will work with a lot of traffic, this mean, a lot of CPU usage, so, keep your rules tiny, just the necessary. Accept all by default just want to make NAT, no more....
 
−
Maybe is good accelerate some ports, (80,443,25,110,21,20,53 etc) remember, every packet running trough our firewall will past trough every rule until it match to one or fall in the pool (default policy).
 
−
 
 
−
===The round robin NAT===
 
−
 
 
−
Let's suppose we have a one ip: 200.aaa.bbb.4 and our gateway is 200.aaa.bbb.1. we can safely put this parameters by default in our configuration. It will not get participation at all in our firewall.
 
  
 
I say I have 3 groups of 10 IPs each to play...... we'll define the NEXT in our firewall script:
 
I say I have 3 groups of 10 IPs each to play...... we'll define the NEXT in our firewall script:
Line 66: Line 53:
 
You can repeat the groups for access, subdivide the networks ETC, iptables make a round robin over the Gr1, Gr2 and Gr3 by default, no modification is needed.
 
You can repeat the groups for access, subdivide the networks ETC, iptables make a round robin over the Gr1, Gr2 and Gr3 by default, no modification is needed.
  
−
It's not necessary create a virtual card (alias) to every IP in the group.
+
It's not necessary to create a virtual card (alias) to every IP in the group.
  
−
It's important that every real router know every group and publish its via BGP (or similar) to the neighbors.
+
It's important that every real router knows every group and publishes its via BGP (or similar) to the neighbours.
  
−
===tips===
+
====tips====
  
 
To accelerate some ports you can put this in the top of FORWARD chain
 
To accelerate some ports you can put this in the top of FORWARD chain
 
  iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
 
  iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
−
  iptables -A FORWARD -p icmp -o eth1 -j ACCEPT
+
  iptables -A FORWARD -p icmp -o eth0 -j ACCEPT
 
  iptables -A FORWARD -p tcp -m multiport --dports 80,443,110,53 -j ACCEPT  # FAST FAST FAST  
 
  iptables -A FORWARD -p tcp -m multiport --dports 80,443,110,53 -j ACCEPT  # FAST FAST FAST  
 
  iptables -A FORWARD -p udp  --dport 53 -j ACCEPT
 
  iptables -A FORWARD -p udp  --dport 53 -j ACCEPT
  
 
This mean:
 
This mean:
−
*the packets incoming will pass only 1 rule if it is a establish connection
+
*the packets incoming will pass only 1 rule if it is an establish connection
−
*the packet incoming will pass 2 rules if is a ping o similar
+
*the packet incoming will pass 2 rules if is a ping or similar
 
*the packet will pass 3 rules if is http, mail or similar
 
*the packet will pass 3 rules if is http, mail or similar
−
*and the DNS recuest will pass 3 o 4 rules until go out
+
*and the DNS request will pass 3 o 4 rules until go out
  
−
The virus outcoming will KILL our machine, and we not need to share "windows" conversatios so, kill them!!!!
+
The outgoing virus will KILL our machine, and we not need to share "windows" conversations so, kill them!!!!
 
   #VIRUS
 
   #VIRUS
 
  iptables -A FORWARD -p tcp --dport 135:139 -j DROP
 
  iptables -A FORWARD -p tcp --dport 135:139 -j DROP
Line 93: Line 80:
 
If you can, before they reach our machine.
 
If you can, before they reach our machine.
  
−
=The Hight Performance=
+
===The High Performance===
  
 
We get to the real important part of this howto.
 
We get to the real important part of this howto.
  
−
In our run to get a really big numbers of host running trough our machine we miss some things
+
In our run to get a really big number of hosts running through our machine we miss some things
−
# We forget that is just one NICs to potentially more than 1000 Mac Addresses. The card shared memory is not prepare to this!!!!!
+
# We forget that is just one NICs to potentially more than 8000 Mac Addresses. The card shared memory is not prepare for this!!!!!
−
# By default iptables is not prepare to make this numbers of connections simultaneously !!!!!!
+
# By default iptables is not prepared to make this number of connections simultaneously !!!!!!
  
 
So...
 
So...
  
−
To the first isue...
+
To the first issue...
−
I get some error messages in the logs realtive to this.... I'm really sorry, I lost this logs and don't remember what they say. But the answer is this... increase the treshold memory to the neighbors.
+
I get some error messages in the logs relative to this, I'm really sorry, I lost these logs and do not remember what they said. But the answer is this, increase the threshold memory to the neighbours.
 
Type this and read:
 
Type this and read:
  
Line 114: Line 101:
 
  1024
 
  1024
  
−
Next you can put this in the /etc/sysctrl
+
Next you can put this in the /etc/sysctrl.conf
 
  net.ipv4.neigh.default.gc_thresh1 = 512
 
  net.ipv4.neigh.default.gc_thresh1 = 512
 
  net.ipv4.neigh.default.gc_thresh2 = 1024
 
  net.ipv4.neigh.default.gc_thresh2 = 1024
 
  net.ipv4.neigh.default.gc_thresh3 = 2048
 
  net.ipv4.neigh.default.gc_thresh3 = 2048
−
and make ''sysctl -p'' to increase to the double!!! with this I get no errors!!!!!
+
and make ''sysctl -p'' to increase to the double!!! (no reboot needed) with this I get no errors!!!!!
  
−
The next will need some comprehension about buckets and conntracks and hashsize (the way how iptables manage the nat connections).
+
The next part will need some comprehension about buckets and conntracks and hashsize (the way how iptables manage the nat connections).
−
There is a very good document about this in [http://www.wallfire.org/misc/netfilter_conntrack_perf.txt here]. Read it!!!!
+
There is a very good document about this at [http://www.wallfire.org/misc/netfilter_conntrack_perf.txt here]. Read it!!!!
 
Some thing are change since IPtables is know as Netfiler.
 
Some thing are change since IPtables is know as Netfiler.
  
 
In resume!!!
 
In resume!!!
 
Put this in your modules section:
 
Put this in your modules section:
−
  MODULES=(8021q 'nf_conntrack hashsize=1048576' nf_conntrack_ftp ...and other nf_stuff .......)
+
  MODULES=(8021q 'nf_conntrack hashsize=1048576' nf_conntrack_ftp  
−
The last ones is just to avoid some problems tha we have with ftp connections (I thing this is no necessary anymore).
+
                                ...and other nf_stuff .......)
−
The ''''nf_conntrack hashsize=1048576'''' increase the numbers of the hashsize (increase the kernel memory designated to NAT connections)
+
The last ones is just to avoid some problems that we have with ftp connections (I thing this is not necessary anymore).
 +
The ''''nf_conntrack hashsize=1048576'''' increase the numbers of the hashsize (increase the kernel memory designated to NAT connections) (need reboot or '''reload module''' :-) see with ''dmesg | grep conntrack'')
  
 
And the next is put some similar to the ''/etc/sysctrl.conf'' file
 
And the next is put some similar to the ''/etc/sysctrl.conf'' file
Line 134: Line 122:
 
  net.netfilter.nf_conntrack_max = 1048576
 
  net.netfilter.nf_conntrack_max = 1048576
 
  ...
 
  ...
−
In my case is the same number, thats mean that I have 1 coneccion for bucket!!!! I don't need more!!!! by default NetFilter put rate of 1:8. I.E. 8 conections per bucket!! (I think, not remmember well)..
+
And do the ''sysctl -p'' command
 +
 
 +
In my case is the same number, that means that I have 1 connection for bucket!!!! I do not need more!!!! by default NetFilter put rate of 1:8. I.E. 8 conections per bucket!! (I think, not remember well)..
  
 
In our case we get about 600.000 simultaneous connections in 2 1Giga NICs cards, You can see this with the next command
 
In our case we get about 600.000 simultaneous connections in 2 1Giga NICs cards, You can see this with the next command
 
  # cat /proc/sys/net/netfilter/nf_conntrack_count
 
  # cat /proc/sys/net/netfilter/nf_conntrack_count
 
And put this in a snmpd agent to get and graph it in a MRTG/cacti server  ..... uuuuuuu homework
 
And put this in a snmpd agent to get and graph it in a MRTG/cacti server  ..... uuuuuuu homework
−
A sample [http://200.112.232.3/conntrack.png here]. (Not actualized, not ever here but I'll try.)
 
  
−
Finally
+
===The iproute2===
−
=The iproute2 part=
+
 
 +
We have 3 big access to Internet!!! This is because we manage 3 class C groups of IPs  (some restrictions of BGP) in this firewall. So, we have 3 incoming traffics that we can manage, but only one outgoing!!! Our default gateway.
 +
This can easily fill our outgoing quote, so we have to spare it.
 +
 
 +
First we have to put some new tables to ''/etc/iproute2/rt_tables'' file
 +
# echo 200 PRO_1 >> /etc/iproute2/rt_tables
 +
# echo 205 PRO_2 >> /etc/iproute2/rt_tables
 +
# echo 210 PRO_3 >> /etc/iproute2/rt_tables
 +
Can be more, can be less, depends on traffic
 +
 
 +
Second we have to give a default gateway to this tables
 +
# ip route add default via 200.aaa.bbb.2 table PRO_1
 +
# ip route add default via 200.aaa.bbb.3 table PRO_2
 +
# ip route add default via 200.aaa.bbb.4 table PRO_3
 +
It's recommended but not necessary put the local interfaces to each table. If you do not put the next few lines you will get not answer of ping in the local network, but you will be able to pass trough.
 +
# ip route add 192.168.0.0/21 via 192.168.0.1 table PRO_1
 +
# ip route add 192.168.8.0/21 via 192.168.8.1 table PRO_1
 +
# ip route add 192.168.15.0/21 via 192.168.15.1 table PRO_1
 +
.....
 +
same PRO_2, same PRO_3
 +
The last thing is to give the order to the incoming packages
 +
# ip rule add from 192.168.0.0/21 table PRO_1
 +
....
 +
....
 +
Again, you can play with the PRO_X and even you can play with the mask and submask
 +
For example we want to give only a one class C to outgoing to PRO_3
 +
# ip rule add from 192.168.1.0/24 table PRO_3
 +
Put this before the <NET>/21

Revision as of 20:35, 30 January 2013


Merge-arrows-2.pngThis article or section is a candidate for merging with Router.Merge-arrows-2.png

Notes: please use the second argument of the template to provide more detailed indications. (Discuss in Talk:High Performance Firewall#)

Tango-edit-clear.pngThis article or section needs language, wiki syntax or style improvements.Tango-edit-clear.png

Reason: please use the first argument of the template to provide a brief explanation. (Discuss in Talk:High Performance Firewall#)
High Performance Firewall / Nat with iptables and VLANs and iproute2

Imagine this, you have more than two networks separated by Virtual Lans protocols (IEEE 802.1q) or VLANs, carried to you by an intelligent/manageable switch on one troncal line 10/100/1000 MB HD/FD (naturally the best is 1000 MB FD).

You have to share internet to a really BIG numbers of hosts, and maintain a good performance. The first choice is to separate the networks into a equal numbers of ports and maybe a more numbers of firewalls machines. This is not really cost effective, but works.

The second one is what I did. The history of how this begin is related to a some emergency/burn/crash/out of a group of Cisco PIXs. I won't go too deeply into that.

VLAN support

The first thing we have to do is give the kernel the capacity to work with Jumbo Frames. This is done by adding the 8021q module to the kernel 

# modprobe 8021q

and/or put in modules in /etc/rc.conf; alternatively using systemd modules are managed in /etc/modules-load.d/

Next we have to create the virtuals NICs with this command vconfig. Let's suppose we have vlans 20,30,40 and 50 working in our core network.

Note: vconfig deprecated, use iproute2 instead
# vconfig add ethX 20
# vconfig add ethX 30
...
# vconfig add ethX 50

Where ethX is the trunk NIC Now, if we want to see the interfaces just put ifconfig -a and we will get a list.

So .... next just set the private address as you want. For example (using eth1 like a troncal) 

# ifconfig eth1.20 192.168.0.1 netmask 255.255.248.0
# ifconfig eth1.30 192.168.8.1 netmask 255.255.248.0
...
 etc.

I will not explain the number of host neither the mask.... I do a kind of daemon with this, a vlan.conf file in /etc and a vland in /etc/rc.d... I can share this if you want?

The round robin NAT

Let's suppose we have a one ip: 200.aaa.bbb.6 and our gateway is 200.aaa.bbb.1. we can safely put these parameters by default in our configuration. It will not get participation at all in our firewall.

I say I have 3 groups of 10 IPs each to play...... we'll define the NEXT in our firewall script: 

Gr1='200.AAA.CCC.10-200.AAA.CCC.20'
Gr2='200.AAA.DDD.10-200.AAA.DDD.20'
Gr3='200.AAA.EEE.10-200.AAA.EEE.20'

And the next important line is: 

iptables -t nat -A POSTROUTING -s 192.168.0.0/21  -j SNAT --to $Gr1 #ACCESS VLAN 10
iptables -t nat -A POSTROUTING -s 192.168.8.0/21  -j SNAT --to $Gr2 #ACCESS VLAN 20
iptables -t nat -A POSTROUTING -s 192.168.15.0/21  -j SNAT --to $Gr1 #ACCESS VLAN 30
.... etc

You can repeat the groups for access, subdivide the networks ETC, iptables make a round robin over the Gr1, Gr2 and Gr3 by default, no modification is needed.

It's not necessary to create a virtual card (alias) to every IP in the group.

It's important that every real router knows every group and publishes its via BGP (or similar) to the neighbours.

tips

To accelerate some ports you can put this in the top of FORWARD chain 

iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -p icmp -o eth0 -j ACCEPT
iptables -A FORWARD -p tcp -m multiport --dports 80,443,110,53 -j ACCEPT  # FAST FAST FAST 
iptables -A FORWARD -p udp  --dport 53 -j ACCEPT

This mean:

  • the packets incoming will pass only 1 rule if it is an establish connection
  • the packet incoming will pass 2 rules if is a ping or similar
  • the packet will pass 3 rules if is http, mail or similar
  • and the DNS request will pass 3 o 4 rules until go out

The outgoing virus will KILL our machine, and we not need to share "windows" conversations so, kill them!!!! 

 #VIRUS
iptables -A FORWARD -p tcp --dport 135:139 -j DROP
iptables -A FORWARD -p tcp --dport 445 -j DROP
iptables -A FORWARD -p udp --dport 135:139 -j DROP
iptables -A FORWARD -p udp --dport 445 -j DROP

If you can, before they reach our machine.

The High Performance

We get to the real important part of this howto.

In our run to get a really big number of hosts running through our machine we miss some things

  1. We forget that is just one NICs to potentially more than 8000 Mac Addresses. The card shared memory is not prepare for this!!!!!
  2. By default iptables is not prepared to make this number of connections simultaneously !!!!!!

So...

To the first issue... I get some error messages in the logs relative to this, I'm really sorry, I lost these logs and do not remember what they said. But the answer is this, increase the threshold memory to the neighbours. Type this and read: 

# cat /proc/sys/net/ipv4/neigh/default/gc_thresh1 
128
# cat /proc/sys/net/ipv4/neigh/default/gc_thresh2 
512
# cat /proc/sys/net/ipv4/neigh/default/gc_thresh3 
1024

Next you can put this in the /etc/sysctrl.conf 

net.ipv4.neigh.default.gc_thresh1 = 512
net.ipv4.neigh.default.gc_thresh2 = 1024
net.ipv4.neigh.default.gc_thresh3 = 2048

and make sysctl -p to increase to the double!!! (no reboot needed) with this I get no errors!!!!!

The next part will need some comprehension about buckets and conntracks and hashsize (the way how iptables manage the nat connections). There is a very good document about this at here. Read it!!!! Some thing are change since IPtables is know as Netfiler.

In resume!!! Put this in your modules section: 

MODULES=(8021q 'nf_conntrack hashsize=1048576' nf_conntrack_ftp 
                               ...and other nf_stuff .......)

The last ones is just to avoid some problems that we have with ftp connections (I thing this is not necessary anymore). The 'nf_conntrack hashsize=1048576' increase the numbers of the hashsize (increase the kernel memory designated to NAT connections) (need reboot or reload module :-) see with dmesg | grep conntrack)

And the next is put some similar to the /etc/sysctrl.conf file 

...
net.netfilter.nf_conntrack_max = 1048576
...

And do the sysctl -p command

In my case is the same number, that means that I have 1 connection for bucket!!!! I do not need more!!!! by default NetFilter put rate of 1:8. I.E. 8 conections per bucket!! (I think, not remember well)..

In our case we get about 600.000 simultaneous connections in 2 1Giga NICs cards, You can see this with the next command 

# cat /proc/sys/net/netfilter/nf_conntrack_count

And put this in a snmpd agent to get and graph it in a MRTG/cacti server ..... uuuuuuu homework

The iproute2

We have 3 big access to Internet!!! This is because we manage 3 class C groups of IPs (some restrictions of BGP) in this firewall. So, we have 3 incoming traffics that we can manage, but only one outgoing!!! Our default gateway. This can easily fill our outgoing quote, so we have to spare it.

First we have to put some new tables to /etc/iproute2/rt_tables file 

# echo 200 PRO_1 >> /etc/iproute2/rt_tables
# echo 205 PRO_2 >> /etc/iproute2/rt_tables
# echo 210 PRO_3 >> /etc/iproute2/rt_tables

Can be more, can be less, depends on traffic

Second we have to give a default gateway to this tables 

# ip route add default via 200.aaa.bbb.2 table PRO_1
# ip route add default via 200.aaa.bbb.3 table PRO_2
# ip route add default via 200.aaa.bbb.4 table PRO_3

It's recommended but not necessary put the local interfaces to each table. If you do not put the next few lines you will get not answer of ping in the local network, but you will be able to pass trough. 

# ip route add 192.168.0.0/21 via 192.168.0.1 table PRO_1
# ip route add 192.168.8.0/21 via 192.168.8.1 table PRO_1
# ip route add 192.168.15.0/21 via 192.168.15.1 table PRO_1
.....
same PRO_2, same PRO_3

The last thing is to give the order to the incoming packages 

# ip rule add from 192.168.0.0/21 table PRO_1
....
....

Again, you can play with the PRO_X and even you can play with the mask and submask For example we want to give only a one class C to outgoing to PRO_3 

# ip rule add from 192.168.1.0/24 table PRO_3

Put this before the <NET>/21

Retrieved from "https://wiki.archlinux.org/index.php?title=High_Performance_Firewall&oldid=245703"