Jumbo frames

From ArchWiki
Jump to: navigation, search

What are Jumbo Frames

The following paragraph was taken from the Wikpedia Jumbo Frame Article: In computer networking, jumbo frames are Ethernet frames with more than 1,500 bytes of payload (MTU). Conventionally, jumbo frames can carry up to 9,000 bytes of payload, but variations exist and some care must be taken when using the term. Many, but not all, Gigabit Ethernet switches and Gigabit Ethernet network interface cards support jumbo frames, but all Fast Ethernet switches and Fast Ethernet network interface cards support only standard-sized frames. Most national research and education networks (such as Internet2/NLR, ESnet, GÉANT, and AARNet) support jumbo frames, but most commercial Internet service providers do not support jumbo frames.

Using a larger MTU value (jumbo frames) can significantly speed up your network transfers. See this webpage for more.

Requirements for Jumbo Frames

  1. Must be GigLAN backbone (i.e. 1000baseT)
  2. Local NICs in the local PCs must support JFs
  3. Switches must support JFs

Real World Examples

It is important to test various frame sizes doing typical file transfers to determine the optimal setting. There is not a one-size-fits-all setting for jumbo frames. In fact, as illustrated below, depending on the hardware, a frame size that is too large can actually hurt network throughput.

Three different frame sizes (standard 1500, jumbo 4000, and jumbo 9000) were tested under several different conditions (single large file and multiple small files). The results indicate that for this particular hardware, a 4000 frame size gave the optimal results. Again, this result is not general; users will need to evaluate the effect that different frame sizes have on different file transfers on a case-by-case basis.

Both machine A and machine B are using approx. 5-year-old EIDE hardrives; a modern SATA II HDD and MB will give higher throughput for sure, but the data are still valid. Also, both machines are using Dlink 530T NICs (PCI bus) and are set to the frame size indicated in the tables below (1500, 4000, and 9000).

Test 1 - Single large file (1,048,522 kb) via Samba

Times and throughput represent the average of three runs

mtu=1500 t (sec) Kb/sec mtu=4000 t (sec) Kb/sec mtu=9000 t (sec) Kb/sec
A to B 48 21,844 A to B 44 23,830 A to B 49 21,398
B to A 81 12,945 B to A 41 25,574 B to A 41 25,574

Summary of Test 1

4k vs. 1.5k  % change 9k vs. 1.5k  % change
A to B +9 % A to B -2 %
B to A +98 % B to A +98 %

Test 2 - Several small files (1,283,439 kb total) via Samba

Times and throughput represent the average of three runs

mtu=1500 t (sec) Kb/sec mtu=4000 t (sec) Kb/sec mtu=9000 t (sec) Kb/sec
A to B 59 21,753 A to B 51 25,165 A to B 57 22,516
B to A 94 13,654 B to A 46 27,901 B to A 49 26,193

Summary of Test 2

4k vs. 1.5k  % change 9k vs. 1.5k  % change
A to B +16 % A to B +4 %
B to A +4% B to A +92 %

Sustained Transfer Speed

An average speed of 26.6 MB/s on a 30+ GB xfer using 4k jumbo frames. The slowest drive in the link was a Western Digital WD3200JB - 5-year-old, EIDE 320 GB (ATA100/7200 RPM/8 MB cache). Burst speeds (i.e. 300-400 meg files) through the network are about 80% of the burst speed drive-to-drive which is not too bad (approx. 30 MB/s network vs. approx 38 MB/s drive-to-drive).

Example LAN Architecture using Jumbo Frames

PC1 (JF Enabled)

PC2 (JF Enabled) <----> GigLAN Switch (JF Enabled) <----> Router (JF Unenabled) <----> Cable modem (JF Unenabled)

PC3 (JF Enabled)

In the above example, all the PCs have NICs set to use JFs and are all connected to a GigLAN switch that can also use JFs. The switch is in turn connect via the uplink port to a router that cannot use jumbo frames which is in turn connected to a cable modem which also cannot use jumbo frames.

Contrary to what some websites state, this setup works 100 % fine. Xfers inside the JF portion on the network (i.e. behind the switch) are very fast. Xfers to the WAN (the Internet) from PCs behind the switch are just as fast as a PC without JFs enabled connected either directly to the cable modem, or to the router.

Using Jumbo Frames on Arch Linux

Invoke ip with the mtu parameter as follows:

# ip link set ethx mtu y

Where ethx is the ethernet adapter in question (eth0, eth1, etc.) and y is the size of the frame you wish to use (1500, 4000, 9000).

Verify that the setting has been applied:

$ ip link show | grep mtu


$ ip link show | grep mtu
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 4000 qdisc pfifo_fast state UP qlen 1000
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000

To make the setting survive a reboot, place a line in /etc/rc.local:

ip link set eth0 mtu 4000