Difference between revisions of "Sysctl"

From ArchWiki
Jump to: navigation, search
(Split Networking and Security sections. Add "Hide kernel symbol addresses" section.)
m (Improving performance: rm expansion flag)
 
(88 intermediate revisions by 28 users not shown)
Line 1: Line 1:
 
{{Lowercase title}}
 
{{Lowercase title}}
 
[[Category:Kernel]]
 
[[Category:Kernel]]
[[Wikipedia:sysctl|sysctl]] is a tool (in [core] package {{Pkg|procps-ng}}) for examining and changing kernel parameters at runtime. sysctl is implemented in procfs, the virtual process file system at {{ic|/proc/}}.
+
[[ja:Sysctl]]
 +
[[Wikipedia:sysctl|sysctl]] is a tool for examining and changing [[kernel parameters]] at runtime (package {{Pkg|procps-ng}} in [[official repositories]]). sysctl is implemented in procfs, the virtual process file system at {{ic|/proc/}}.
  
 
== Configuration ==
 
== Configuration ==
  
The '''sysctl''' preload/configuration file is located at {{ic|/etc/sysctl.conf}}.
+
{{Note|From version 207 and 21x, [[systemd]] only applies settings from {{Ic|/etc/sysctl.d/*.conf}} and {{Ic|/usr/lib/sysctl.d/*.conf}}. If you had customized {{Ic|/etc/sysctl.conf}}, you need to rename it as {{Ic|/etc/sysctl.d/99-sysctl.conf}}. If you had e.g. {{Ic|/etc/sysctl.d/foo}}, you need to rename it to {{Ic|/etc/sysctl.d/foo.conf}}.}}
  
{{hc|/etc/sysctl.conf
+
The '''sysctl''' preload/configuration file can be created at {{Ic|/etc/sysctl.d/99-sysctl.conf}}. For [[systemd]], {{Ic|/etc/sysctl.d/}} and {{Ic|/usr/lib/sysctl.d/}} are drop-in directories for kernel sysctl parameters. The naming and source directory decide the order of processing, which is important since the last parameter processed may override earlier ones. For example, parameters in a {{ic|/usr/lib/sysctl.d/50-default.conf}} will be overriden by equal parameters in {{ic|/etc/sysctl.d/50-default.conf}} and any configuration file processed later from both directories.  
|2=<nowiki>
 
# /etc/sysctl.conf - Configuration file for setting system variables
 
# See sysctl.conf (5) for information.
 
  
# you can have the CD-ROM close when you use it, and open
+
To load all configuration files manually, execute
# when you are done.
+
# sysctl --system
#dev.cdrom.autoeject = 1
+
which will also output the applied hierarchy. A single parameter file can also be loaded explicitly with
#dev.cdrom.autoclose = 1
+
# sysctl -p ''filename.conf''
  
# protection from the SYN flood attack
+
See [http://0pointer.de/blog/projects/the-new-configuration-files the new configuration files] and more specifically {{man|5|sysctl.d}} for more information.
net.ipv4.tcp_syncookies = 1
+
 
 +
The parameters available are those listed under {{Ic|/proc/sys/}}. For example, the {{Ic|kernel.sysrq}} parameter refers to the file {{Ic|/proc/sys/kernel/sysrq}} on the file system. The {{Ic|sysctl -a}} command can be used to display all currently available values.
  
# see the evil packets in your log files
+
{{Note|If you have the kernel documentation installed ({{Pkg|linux-docs}}), you can find detailed information about sysctl settings in {{Ic|/usr/lib/modules/$(uname -r)/build/Documentation/sysctl/}}. It is highly recommended reading these before changing sysctl settings.}}
#net.ipv4.conf.all.log_martians = 1
 
  
# if not functioning as a router, there is no need to accept redirects or source routes
+
Settings can be changed through file manipulation or using the {{Ic|sysctl}} utility. For example, to temporarily enable the [[Wikipedia:Magic_SysRq_key|magic SysRq key]]:
#net.ipv4.conf.all.accept_redirects = 0
 
#net.ipv4.conf.all.accept_source_route = 0
 
#net.ipv6.conf.all.accept_redirects = 0
 
#net.ipv6.conf.all.accept_source_route = 0
 
  
# Disable packet forwarding
+
# sysctl kernel.sysrq=1
net.ipv4.ip_forward = 0
 
net.ipv6.conf.all.forwarding = 0
 
  
# Enable IPv6 Privacy Extensions
+
or:
net.ipv6.conf.default.use_tempaddr = 2
 
net.ipv6.conf.all.use_tempaddr = 2
 
  
# sets the port range used for outgoing connections
+
# echo "1" > /proc/sys/kernel/sysrq
#net.ipv4.ip_local_port_range = 32768    61000
 
  
# Swapping too much or not enough? Disks spinning up when you'd
+
To preserve changes between reboots, add or modify the appropriate lines in {{Ic|/etc/sysctl.d/99-sysctl.conf}} or another applicable parameter file in {{Ic|/etc/sysctl.d/}}.
# rather they didn't? Tweak these.
+
{{Tip|Some parameters that can be applied may depend on kernel modules which in turn might not be loaded. For example parameters in {{ic|/proc/sys/net/bridge/*}} depend on the {{ic|br_netfilter}} module. If it is not loaded at runtime (or after a reboot), those will ''silently'' not be applied. See [[Kernel modules]].}}
#vm.vfs_cache_pressure = 100
 
#vm.laptop_mode = 0
 
#vm.swappiness = 60
 
  
#kernel.printk_ratelimit_burst = 10
+
== Security ==
#kernel.printk_ratelimit = 5
 
#kernel.panic_on_oops = 0
 
  
# Reboot 600 seconds after a panic
+
See [[Security#Kernel hardening]].
#kernel.panic = 600
 
  
# Disable SysRq key (note: console security issues)
+
== Networking ==
kernel.sysrq = 0
 
</nowiki>}}
 
  
The parameters available are those listed under {{ic|/proc/sys/}}. For example, the {{Ic|kernel.sysrq}} parameter refers to the file {{ic|/proc/sys/kernel/sysrq}} on the file system. The {{Ic|sysctl -a}} command can be used to display all values currently available.
+
=== Improving performance ===
  
{{Note|If you have the kernel documentation installed ({{Pkg|linux-docs}}), you can find detailed information about sysctl settings in {{ic|/usr/src/linux-$(uname -r)/Documentation/sysctl/}}. It is highly recommended reading before changing sysctl settings.}}
+
{{Style|Comments don't belong in the code box, use the wiki markup.}}
  
Settings can be changed through file manipulation or using the sysctl utility. For example, to temporarily enable the [[Wikipedia:Magic_SysRq_key|magic sysrq key]]:
+
# The maximum size of the receive queue.
 +
# The received frames will be stored in this queue after taking them from the ring buffer on the NIC.
 +
# Use high value for high speed cards to prevent loosing packets.
 +
# In real time application like SIP router, long queue must be assigned with high speed CPU otherwise the data in the queue will be out of date (old).
 +
net.core.netdev_max_backlog = 65536
  
  # sysctl kernel.sysrq=1
+
  # The maximum ancillary buffer size allowed per socket.
 +
# Ancillary data is a sequence of struct cmsghdr structures with appended data.
 +
net.core.optmem_max = 65536
  
or:
+
# The upper limit on the value of the backlog parameter passed to the listen function.
 +
# Setting to higher values is only needed on a single highloaded server where new connection rate is high/bursty
 +
net.core.somaxconn = 16384
  
  # echo "1" > /proc/sys/kernel/sysrq
+
  # The default and maximum amount for the receive/send socket memory
 +
# By default the Linux network stack is not configured for high speed large file transfer across WAN links.
 +
# This is done to save memory resources.
 +
# You can easily tune Linux network stack by increasing network buffers size for high-speed networks that connect server systems to handle more network packets.
 +
net.core.rmem_default = 1048576
 +
net.core.wmem_default = 1048576
 +
net.core.rmem_max = 16777216
 +
net.core.wmem_max = 16777216
 +
net.ipv4.tcp_rmem = 4096 87380 16777216
 +
net.ipv4.tcp_wmem = 4096 65536 16777216
 +
net.ipv4.udp_rmem_min = 16384
 +
net.ipv4.udp_wmem_min = 16384
  
To preserve changes between reboots, add or modify the appropriate lines in {{ic|/etc/sysctl.conf}}.
+
# An extension to the transmission control protocol (TCP) that helps reduce network latency by enabling data to be exchanged during the sender’s initial TCP SYN.
 +
# If both of your server and client are deployed on Linux 3.7.1 or higher, you can turn on fast_open for lower latency
 +
net.ipv4.tcp_fastopen = 3
  
{{Tip|After changing settings in {{ic|/etc/sysctl.conf}}, you can load them with
+
# The maximum queue length of pending connections 'Waiting Acknowledgment'
  # sysctl -p
+
# In the event of a synflood DOS attack, this queue can fill up pretty quickly, at which point tcp_syncookies will kick in allowing your system to continue to respond to legitimate traffic, and allowing you to gain access to block malicious IPs.
}}
+
  # If the server suffers from overloads at peak times, you may want to increase this value a little bit.
 +
net.ipv4.tcp_max_syn_backlog = 65536
  
== systemd ==
+
# The maximum number of sockets in 'TIME_WAIT' state.
If you have [[systemd]] installed, you will find {{ic|/etc/sysctl.d/}} which is ''"a drop-in directory for kernel sysctl parameters, extending what you can already do with /etc/sysctl.conf."'' See [http://0pointer.de/blog/projects/the-new-configuration-files The New Configuration Files] and more specifically [http://0pointer.de/public/systemd-man/sysctl.d.html systemd's sysctl.d man page] for more information.
+
  # After reaching this number the system will start destroying the socket in this state.
 +
# Increase this to prevent simple DOS attacks
 +
net.ipv4.tcp_max_tw_buckets = 65536
  
== Security ==
+
# Whether TCP should start at the default window size only for new connections or also for existing connections that have been idle for too long.
 +
# It kills persistent single connection performance and should be turned off.
 +
net.ipv4.tcp_slow_start_after_idle = 0
  
===Preventing link [[Wikipedia:TOCTOU|TOCTOU]] vulnerabilities===
+
# Whether TCP should reuse an existing connection in the TIME-WAIT state for a new outgoing connection if the new timestamp is strictly bigger than the most recent timestamp recorded for the previous connection.
 +
# This helps avoid from running out of available network sockets.
 +
net.ipv4.tcp_tw_reuse = 1
  
See the [https://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commitdiff;h=800179c9b8a1e796e441674776d11cd4c05d61d7 commit message] for when this feature was added for the rationale.
+
# Fast-fail FIN connections which are useless.
 +
net.ipv4.tcp_fin_timeout = 15
  
{{bc|1=
+
# TCP keepalive is a mechanism for TCP connections that help to determine whether the other end has stopped responding or not.
fs.protected_hardlinks = 1
+
# TCP will send the keepalive probe contains null data to the network peer several times after a period of idle time. If the peer does not respond, the socket will be closed automatically.
fs.protected_symlinks = 1
+
# By default, TCP keepalive process waits for two hours (7200 secs) for socket activity before sending the first keepalive probe, and then resend it every 75 seconds. As long as there is TCP/IP socket communications going on and active, no keepalive packets are needed.
}}
+
# With the following settings, your application will detect dead TCP connections after 120 seconds (60s + 10s + 10s + 10s + 10s + 10s + 10s)
 +
net.ipv4.tcp_keepalive_time = 60
 +
net.ipv4.tcp_keepalive_intvl = 10
 +
net.ipv4.tcp_keepalive_probes = 6
  
===Hide kernel symbol addresses===
+
# The longer the MTU the better for performance, but the worse for reliability.
 +
# This is because a lost packet means more data to be retransmitted and because many routers on the Internet can't deliver very long packets.
 +
# Enable smart MTU discovery when an ICMP black hole detected.
 +
net.ipv4.tcp_mtu_probing = 1
  
Enabling {{ic|kernel.kptr_restrict}} will hide kernel symbol addresses in {{ic|/proc/kallsyms}} from regular users, making it more difficult for kernel exploits to resolve addresses/symbols dynamically. This will not help that much on a precompiled Arch Linux kernel, since a determined attacker could just download the kernel package and get the symbols manually from there, but if you're compiling your own kernel, this can help mitigating local root exploits. This will break some {{Pkg|perf}} commands when used by non-root users (but main {{Pkg|perf}} features require root access anyway). See [https://bugs.archlinux.org/task/34323 this bug] for more information.
+
# Turn timestamps off to reduce performance spikes related to timestamp generation.
 +
net.ipv4.tcp_timestamps = 0
  
{{bc|1=
+
*You can use http://www.speedtest.net to benchmark internetwork performance before/after the change
kernel.kptr_restrict = 1
 
}}
 
  
== Networking ==
+
=== TCP/IP stack hardening ===
  
===Improving Performance===
+
The following specifies a parameter set to tighten network security options of the kernel for the IPv4 protocol and related IPv6 parameters where an equivalent exists.  
{{Warning|This may cause dropped frames with load-balancing and NATs, only use this for a server that communicates only over your local network.}}
 
{{bc|1=
 
# reuse/recycle time-wait sockets
 
net.ipv4.tcp_tw_reuse = 1
 
net.ipv4.tcp_tw_recycle = 1
 
}}
 
  
=== TCP/IP stack hardening ===
+
For some usecases, for example using the system as a [[router]], other parameters may be useful or required as well.
  
{{bc|1=
+
{{hc|/etc/sysctl.d/51-net.conf|2=
#### ipv4 networking ####
+
#### ipv4 networking and equivalent ipv6 parameters ####
  
## TCP SYN cookie protection
+
## TCP SYN cookie protection (default)
 
## helps protect against SYN flood attacks
 
## helps protect against SYN flood attacks
 
## only kicks in when net.ipv4.tcp_max_syn_backlog is reached
 
## only kicks in when net.ipv4.tcp_max_syn_backlog is reached
Line 121: Line 130:
 
## (not widely supported outside of linux, but conforms to RFC)
 
## (not widely supported outside of linux, but conforms to RFC)
 
net.ipv4.tcp_rfc1337 = 1
 
net.ipv4.tcp_rfc1337 = 1
 +
 +
## sets the kernels reverse path filtering mechanism to value 1 (on)
 +
## will do source validation of the packet's recieved from all the interfaces on the machine
 +
## protects from attackers that are using ip spoofing methods to do harm
 +
net.ipv4.conf.default.rp_filter = 1
 +
net.ipv4.conf.all.rp_filter = 1
  
 
## tcp timestamps
 
## tcp timestamps
Line 129: Line 144:
 
net.ipv4.tcp_timestamps = 0
 
net.ipv4.tcp_timestamps = 0
 
#net.ipv4.tcp_timestamps = 1
 
#net.ipv4.tcp_timestamps = 1
 
## source address verification (sanity checking)
 
## helps protect against spoofing attacks
 
net.ipv4.conf.all.rp_filter = 1
 
 
## disable ALL packet forwarding (not a router, disable it)
 
net.ipv4.ip_forward = 0
 
  
 
## log martian packets
 
## log martian packets
 +
net.ipv4.conf.default.log_martians = 1
 
net.ipv4.conf.all.log_martians = 1
 
net.ipv4.conf.all.log_martians = 1
  
## ignore echo broadcast requests to prevent being part of smurf attacks
+
## ignore echo broadcast requests to prevent being part of smurf attacks (default)
 
net.ipv4.icmp_echo_ignore_broadcasts = 1
 
net.ipv4.icmp_echo_ignore_broadcasts = 1
  
## optionally, ignore all echo requests
+
## ignore bogus icmp errors (default)
## this is NOT recommended, as it ignores echo requests on localhost as well
 
#net.ipv4.icmp_echo_ignore_all = 1
 
 
 
## ignore bogus icmp errors
 
 
net.ipv4.icmp_ignore_bogus_error_responses = 1
 
net.ipv4.icmp_ignore_bogus_error_responses = 1
 
## IP source routing (insecure, disable it)
 
net.ipv4.conf.all.accept_source_route = 0
 
  
 
## send redirects (not a router, disable it)
 
## send redirects (not a router, disable it)
 +
net.ipv4.conf.default.send_redirects = 0
 
net.ipv4.conf.all.send_redirects = 0
 
net.ipv4.conf.all.send_redirects = 0
  
 
## ICMP routing redirects (only secure)
 
## ICMP routing redirects (only secure)
net.ipv4.conf.all.accept_redirects = 0
+
#net.ipv4.conf.default.secure_redirects = 1 (default)
net.ipv4.conf.all.secure_redirects = 1
+
#net.ipv4.conf.all.secure_redirects = 1 (default)
 +
net.ipv4.conf.default.accept_redirects=0
 +
net.ipv4.conf.all.accept_redirects=0
 +
net.ipv6.conf.default.accept_redirects=0
 +
net.ipv6.conf.all.accept_redirects=0
 +
}}
 +
 
 +
== Virtual memory ==
 +
 
 +
There are several key parameters to tune the operation of the virtual memory (VM) subsystem of the Linux kernel and the write out of dirty data to disk. See the official [https://www.kernel.org/doc/Documentation/sysctl/vm.txt Linux kernel documentation] for more information. For example:
 +
 
 +
* {{ic|1=vm.dirty_ratio = 3}}
 +
: Contains, as a percentage of total available memory that contains free pages and reclaimable pages, the number of pages at which a process which is generating disk writes will itself start writing out dirty data.
 +
 
 +
* {{ic|1=vm.dirty_background_ratio = 2}}
 +
: Contains, as a percentage of total available memory that contains free pages and reclaimable pages, the number of pages at which the background kernel flusher threads will start writing out dirty data.
 +
 
 +
As noted in the comments for the parameters, one needs to consider the total amount of RAM when setting these values. For example, simplifying by taking the installed system RAM instead of available memory:
 +
 
 +
* Consensus is that setting {{ic|vm.dirty_ratio}} to 10% of RAM is a sane value if RAM is say 1 GB (so 10% is {{#expr: 1000/10 round 0}} MB). But if the machine has much more RAM, say 16 GB (10% is {{#expr: 16/10 round 1 }} GB), the percentage may be out of proportion as it becomes several seconds of writeback on spinning disks. A more sane value in this case is 3 (3% of 16 GB is approximately 491 MB).
 +
* Similarly, setting {{ic|vm.dirty_background_ratio}} to 5 may be just fine for small memory values, but again, consider and adjust accordingly for the amount of RAM on a particular system.
 +
 
 +
Another parameter is:
 +
 
 +
* {{ic|1=vm.vfs_cache_pressure = 60}}
 +
: The value controls the tendency of the kernel to reclaim the memory which is used for caching of directory and inode objects (VFS cache). Lowering it from the default value of 100 makes the kernel less inclined to reclaim VFS cache (do not set it to 0, this may produce out-of-memory conditions).
 +
 
 +
== MDADM ==
 +
 
 +
When the kernel performs a resync operation of a software raid device it tries not to create a high system load by restricting the speed of the operation. Using sysctl it is possible to change the lower and upper speed limit.
 +
 
 +
{{bc|1=
 +
# Set maximum and minimum speed of raid resyncing operations
 +
dev.raid.speed_limit_max = 10000
 +
dev.raid.speed_limit_min = 1000
 
}}
 
}}
 +
 +
If mdadm is compiled as a module {{ic|md_mod}}, the above settings are available only after the module has been loaded. If the settings shall be loaded on boot via {{ic|/etc/sysctl.d}}, the module {{ic|md_mod}} may be loaded beforehand through {{ic|/etc/modules-load.d}}.
  
 
== Troubleshooting ==
 
== Troubleshooting ==
 +
 
=== Small periodic system freezes ===
 
=== Small periodic system freezes ===
  
Set dirty bytes to small enough value (for example 4M)
+
Set dirty bytes to small enough value (for example 4M):
  
  vm.dirty_background_bytes = 4194304
+
vm.dirty_background_bytes = 4194304
  vm.dirty_bytes = 4194304
+
vm.dirty_bytes = 4194304
  
Try to change kernel.io_delay_type (x86 only):
+
Try to change {{ic|kernel.io_delay_type}} (x86 only):
 
* 0 - IO_DELAY_TYPE_0X80
 
* 0 - IO_DELAY_TYPE_0X80
 
* 1 - IO_DELAY_TYPE_0XED
 
* 1 - IO_DELAY_TYPE_0XED
Line 176: Line 216:
  
 
== See also ==
 
== See also ==
* The sysctl(8) and sysctl.conf(5) man pages
+
 
* Linux kernel documentation ({{ic|&lt;kernel source dir&gt;/Documentation/sysctl/}})
+
* {{man|8|sysctl}} and {{man|5|sysctl.conf}}
* [http://gotux.net/arch-linux/sysctl-config/ SysCtl Config Tutorial]
+
* [https://www.kernel.org/doc/Documentation/sysctl/ Linux kernel documentation for /proc/sys/]
 +
* Kernel Documentation: [https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt IP Sysctl]
 +
* [http://blog.gotux.net/code/config/sysctl/ SysCtl.conf Tweaked for Security and Cable Speed]{{Dead link|2015|12|14}}
 +
* [http://tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.kernel.html Kernel network parameters for sysctl]

Latest revision as of 11:03, 3 September 2017

sysctl is a tool for examining and changing kernel parameters at runtime (package procps-ng in official repositories). sysctl is implemented in procfs, the virtual process file system at /proc/.

Configuration

Note: From version 207 and 21x, systemd only applies settings from /etc/sysctl.d/*.conf and /usr/lib/sysctl.d/*.conf. If you had customized /etc/sysctl.conf, you need to rename it as /etc/sysctl.d/99-sysctl.conf. If you had e.g. /etc/sysctl.d/foo, you need to rename it to /etc/sysctl.d/foo.conf.

The sysctl preload/configuration file can be created at /etc/sysctl.d/99-sysctl.conf. For systemd, /etc/sysctl.d/ and /usr/lib/sysctl.d/ are drop-in directories for kernel sysctl parameters. The naming and source directory decide the order of processing, which is important since the last parameter processed may override earlier ones. For example, parameters in a /usr/lib/sysctl.d/50-default.conf will be overriden by equal parameters in /etc/sysctl.d/50-default.conf and any configuration file processed later from both directories.

To load all configuration files manually, execute

# sysctl --system 

which will also output the applied hierarchy. A single parameter file can also be loaded explicitly with

# sysctl -p filename.conf

See the new configuration files and more specifically sysctl.d(5) for more information.

The parameters available are those listed under /proc/sys/. For example, the kernel.sysrq parameter refers to the file /proc/sys/kernel/sysrq on the file system. The sysctl -a command can be used to display all currently available values.

Note: If you have the kernel documentation installed (linux-docs), you can find detailed information about sysctl settings in /usr/lib/modules/$(uname -r)/build/Documentation/sysctl/. It is highly recommended reading these before changing sysctl settings.

Settings can be changed through file manipulation or using the sysctl utility. For example, to temporarily enable the magic SysRq key:

# sysctl kernel.sysrq=1

or:

# echo "1" > /proc/sys/kernel/sysrq

To preserve changes between reboots, add or modify the appropriate lines in /etc/sysctl.d/99-sysctl.conf or another applicable parameter file in /etc/sysctl.d/.

Tip: Some parameters that can be applied may depend on kernel modules which in turn might not be loaded. For example parameters in /proc/sys/net/bridge/* depend on the br_netfilter module. If it is not loaded at runtime (or after a reboot), those will silently not be applied. See Kernel modules.

Security

See Security#Kernel hardening.

Networking

Improving performance

Tango-edit-clear.pngThis article or section needs language, wiki syntax or style improvements.Tango-edit-clear.png

Reason: Comments don't belong in the code box, use the wiki markup. (Discuss in Talk:Sysctl#)
# The maximum size of the receive queue.
# The received frames will be stored in this queue after taking them from the ring buffer on the NIC.
# Use high value for high speed cards to prevent loosing packets.
# In real time application like SIP router, long queue must be assigned with high speed CPU otherwise the data in the queue will be out of date (old).
net.core.netdev_max_backlog = 65536
# The maximum ancillary buffer size allowed per socket.
# Ancillary data is a sequence of struct cmsghdr structures with appended data.
net.core.optmem_max = 65536
# The upper limit on the value of the backlog parameter passed to the listen function.
# Setting to higher values is only needed on a single highloaded server where new connection rate is high/bursty
net.core.somaxconn = 16384
# The default and maximum amount for the receive/send socket memory
# By default the Linux network stack is not configured for high speed large file transfer across WAN links.
# This is done to save memory resources.
# You can easily tune Linux network stack by increasing network buffers size for high-speed networks that connect server systems to handle more network packets.
net.core.rmem_default = 1048576
net.core.wmem_default = 1048576
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.udp_rmem_min = 16384
net.ipv4.udp_wmem_min = 16384
# An extension to the transmission control protocol (TCP) that helps reduce network latency by enabling data to be exchanged during the sender’s initial TCP SYN.
# If both of your server and client are deployed on Linux 3.7.1 or higher, you can turn on fast_open for lower latency
net.ipv4.tcp_fastopen = 3
# The maximum queue length of pending connections 'Waiting Acknowledgment'
# In the event of a synflood DOS attack, this queue can fill up pretty quickly, at which point tcp_syncookies will kick in allowing your system to continue to respond to legitimate traffic, and allowing you to gain access to block malicious IPs.
# If the server suffers from overloads at peak times, you may want to increase this value a little bit.
net.ipv4.tcp_max_syn_backlog = 65536
# The maximum number of sockets in 'TIME_WAIT' state.
# After reaching this number the system will start destroying the socket in this state.
# Increase this to prevent simple DOS attacks
net.ipv4.tcp_max_tw_buckets = 65536
# Whether TCP should start at the default window size only for new connections or also for existing connections that have been idle for too long.
# It kills persistent single connection performance and should be turned off.
net.ipv4.tcp_slow_start_after_idle = 0
# Whether TCP should reuse an existing connection in the TIME-WAIT state for a new outgoing connection if the new timestamp is strictly bigger than the most recent timestamp recorded for the previous connection.
# This helps avoid from running out of available network sockets.
net.ipv4.tcp_tw_reuse = 1
# Fast-fail FIN connections which are useless.
net.ipv4.tcp_fin_timeout = 15
# TCP keepalive is a mechanism for TCP connections that help to determine whether the other end has stopped responding or not.
# TCP will send the keepalive probe contains null data to the network peer several times after a period of idle time. If the peer does not respond, the socket will be closed automatically.
# By default, TCP keepalive process waits for two hours (7200 secs) for socket activity before sending the first keepalive probe, and then resend it every 75 seconds. As long as there is TCP/IP socket communications going on and active, no keepalive packets are needed.
# With the following settings, your application will detect dead TCP connections after 120 seconds (60s + 10s + 10s + 10s + 10s + 10s + 10s)
net.ipv4.tcp_keepalive_time = 60
net.ipv4.tcp_keepalive_intvl = 10
net.ipv4.tcp_keepalive_probes = 6
# The longer the MTU the better for performance, but the worse for reliability.
# This is because a lost packet means more data to be retransmitted and because many routers on the Internet can't deliver very long packets.
# Enable smart MTU discovery when an ICMP black hole detected.
net.ipv4.tcp_mtu_probing = 1
# Turn timestamps off to reduce performance spikes related to timestamp generation.
net.ipv4.tcp_timestamps = 0

TCP/IP stack hardening

The following specifies a parameter set to tighten network security options of the kernel for the IPv4 protocol and related IPv6 parameters where an equivalent exists.

For some usecases, for example using the system as a router, other parameters may be useful or required as well.

/etc/sysctl.d/51-net.conf
#### ipv4 networking and equivalent ipv6 parameters ####

## TCP SYN cookie protection (default)
## helps protect against SYN flood attacks
## only kicks in when net.ipv4.tcp_max_syn_backlog is reached
net.ipv4.tcp_syncookies = 1

## protect against tcp time-wait assassination hazards
## drop RST packets for sockets in the time-wait state
## (not widely supported outside of linux, but conforms to RFC)
net.ipv4.tcp_rfc1337 = 1

## sets the kernels reverse path filtering mechanism to value 1 (on)
## will do source validation of the packet's recieved from all the interfaces on the machine
## protects from attackers that are using ip spoofing methods to do harm
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1

## tcp timestamps
## + protect against wrapping sequence numbers (at gigabit speeds)
## + round trip time calculation implemented in TCP
## - causes extra overhead and allows uptime detection by scanners like nmap
## enable @ gigabit speeds
net.ipv4.tcp_timestamps = 0
#net.ipv4.tcp_timestamps = 1

## log martian packets
net.ipv4.conf.default.log_martians = 1
net.ipv4.conf.all.log_martians = 1

## ignore echo broadcast requests to prevent being part of smurf attacks (default)
net.ipv4.icmp_echo_ignore_broadcasts = 1

## ignore bogus icmp errors (default)
net.ipv4.icmp_ignore_bogus_error_responses = 1

## send redirects (not a router, disable it)
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.all.send_redirects = 0

## ICMP routing redirects (only secure)
#net.ipv4.conf.default.secure_redirects = 1 (default)
#net.ipv4.conf.all.secure_redirects = 1 (default)
net.ipv4.conf.default.accept_redirects=0
net.ipv4.conf.all.accept_redirects=0
net.ipv6.conf.default.accept_redirects=0
net.ipv6.conf.all.accept_redirects=0

Virtual memory

There are several key parameters to tune the operation of the virtual memory (VM) subsystem of the Linux kernel and the write out of dirty data to disk. See the official Linux kernel documentation for more information. For example:

  • vm.dirty_ratio = 3
Contains, as a percentage of total available memory that contains free pages and reclaimable pages, the number of pages at which a process which is generating disk writes will itself start writing out dirty data.
  • vm.dirty_background_ratio = 2
Contains, as a percentage of total available memory that contains free pages and reclaimable pages, the number of pages at which the background kernel flusher threads will start writing out dirty data.

As noted in the comments for the parameters, one needs to consider the total amount of RAM when setting these values. For example, simplifying by taking the installed system RAM instead of available memory:

  • Consensus is that setting vm.dirty_ratio to 10% of RAM is a sane value if RAM is say 1 GB (so 10% is 100 MB). But if the machine has much more RAM, say 16 GB (10% is 1.6 GB), the percentage may be out of proportion as it becomes several seconds of writeback on spinning disks. A more sane value in this case is 3 (3% of 16 GB is approximately 491 MB).
  • Similarly, setting vm.dirty_background_ratio to 5 may be just fine for small memory values, but again, consider and adjust accordingly for the amount of RAM on a particular system.

Another parameter is:

  • vm.vfs_cache_pressure = 60
The value controls the tendency of the kernel to reclaim the memory which is used for caching of directory and inode objects (VFS cache). Lowering it from the default value of 100 makes the kernel less inclined to reclaim VFS cache (do not set it to 0, this may produce out-of-memory conditions).

MDADM

When the kernel performs a resync operation of a software raid device it tries not to create a high system load by restricting the speed of the operation. Using sysctl it is possible to change the lower and upper speed limit.

# Set maximum and minimum speed of raid resyncing operations
dev.raid.speed_limit_max = 10000
dev.raid.speed_limit_min = 1000

If mdadm is compiled as a module md_mod, the above settings are available only after the module has been loaded. If the settings shall be loaded on boot via /etc/sysctl.d, the module md_mod may be loaded beforehand through /etc/modules-load.d.

Troubleshooting

Small periodic system freezes

Set dirty bytes to small enough value (for example 4M):

vm.dirty_background_bytes = 4194304
vm.dirty_bytes = 4194304

Try to change kernel.io_delay_type (x86 only):

  • 0 - IO_DELAY_TYPE_0X80
  • 1 - IO_DELAY_TYPE_0XED
  • 2 - IO_DELAY_TYPE_UDELAY
  • 3 - IO_DELAY_TYPE_NONE

See also