[Read This Fine Material] from Joshua Hoblitt

Dynamically changing the MTU of a Linux bridge interface


It appears that at least with the EL6.x 2.6.32-220.23.1.el6.x86_64 kernel and the bridge-utils-1.2-10.el6.x86_64 package, there are restrictions on the possible MTU values that made be set for a bridge interface. Specifically, you may not set the MTU of the bridge group to be greater than the lowest MTU of a componet interface.

Consider the following, fairly typical for a hypervisor node, configuration of a physical interface (eth0), a 802.1q tagged interface (eth0.27), a bridge interface (br27), and a tun interface for a QEMU VM (vnet1). The tagged interface is a virtual interface of the physical ethernet device (eth0). While the tagged interface and tun interface are both members of the br27 bridging group.

Example of a typical Linux bridge group configuration

Example of a typical Linux bridge group configuration

In this example setup, the eth0 and eth0.1 interfaces start off with an MTU of 9000, while br27 and vnet1 are set to an MTU of 1500. The goal is to change the MTU of the tun interface to 9000. What I discovered the hard way is that the order of operations is indeed significant.

Confirmation that the MTU “path” from the physical interface to the bridge group will allow “jumbo” frames:

# ip link show eth0
2: eth0:  mtu 9000 qdisc mq state UP qlen 1000
    link/ether 00:25:90:60:ae:aa brd ff:ff:ff:ff:ff:ff
# ip link show eth0.27
7: eth0.27@eth0:  mtu 9000 qdisc noqueue state UP 
    link/ether 00:25:90:60:ae:aa brd ff:ff:ff:ff:ff:ff
# brctl show br27
bridge name	bridge id		STP enabled	interfaces
br27		8000.00259060aeaa	no		eth0.27

However, we get a rather misleading error when trying to change the MTU on the bridge interface:

# ip link set br27 mtu 9000
RTNETLINK answers: Invalid argument
# ifconfig eth0.27 mtu 9000
SIOCSIFMTU: Numerical result out of range

After some experimental thrashing, I was finally able to change the MTU on the bridge group but only if all interfaces in the bridge group had their MTU increased before changing the MTU of the bridge group itself.

# ip link show vnet1
306: vnet1:  mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500
    link/ether fe:52:00:6e:57:fd brd ff:ff:ff:ff:ff:ff
# ip link set vnet1 mtu 9000
# ip link set br27 mtu 9000
# ip link show br27
8: br27:  mtu 9000 qdisc noqueue state UNKNOWN 
    link/ether 00:25:90:60:ae:aa brd ff:ff:ff:ff:ff:ff

While this behaviour isn’t an unreasonable “safety restriction”, it’s rather non-intuitive to someone familiar with layer 2 networking equipment. Typically, the expectation is that every device on the same L2 segment needs to have it’s MTU manually adjusted and this setting is not transparently coupled with other devices. The confusion is further compounded by a not only unhelpful but actively misleading error messages from both the ip and ifconfig utils.

Leave a Reply