It appears that at least with the EL6.x 2.6.32-220.23.1.el6.x86_64
kernel and the bridge-utils-1.2-10.el6.x86_64
package, there are restrictions on the possible MTU values that made be set for a bridge
interface. Specifically, you may not set the MTU of the bridge group to be greater than the lowest MTU of a componet interface.
Consider the following, fairly typical for a hypervisor node, configuration of a physical interface (eth0
), a 802.1q tagged interface (eth0.27
), a bridge interface (br27
), and a tun
interface for a QEMU VM (vnet1
). The tagged interface is a virtual interface of the physical ethernet device (eth0
). While the tagged interface and tun
interface are both members of the br27
bridging group.
In this example setup, the
eth0
and eth0.1
interfaces start off with an MTU of 9000, while br27
and vnet1
are set to an MTU of 1500. The goal is to change the MTU of the tun
interface to 9000. What I discovered the hard way is that the order of operations is indeed significant.
Confirmation that the MTU “path” from the physical interface to the bridge group will allow “jumbo” frames:
# ip link show eth0 2: eth0:mtu 9000 qdisc mq state UP qlen 1000 link/ether 00:25:90:60:ae:aa brd ff:ff:ff:ff:ff:ff # ip link show eth0.27 7: eth0.27@eth0: mtu 9000 qdisc noqueue state UP link/ether 00:25:90:60:ae:aa brd ff:ff:ff:ff:ff:ff # brctl show br27 bridge name bridge id STP enabled interfaces br27 8000.00259060aeaa no eth0.27 vnet1
However, we get a rather misleading error when trying to change the MTU on the bridge interface:
# ip link set br27 mtu 9000 RTNETLINK answers: Invalid argument # ifconfig eth0.27 mtu 9000 SIOCSIFMTU: Numerical result out of range
After some experimental thrashing, I was finally able to change the MTU on the bridge group but only if all interfaces in the bridge group had their MTU increased before changing the MTU of the bridge group itself.
# ip link show vnet1 306: vnet1:mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500 link/ether fe:52:00:6e:57:fd brd ff:ff:ff:ff:ff:ff # ip link set vnet1 mtu 9000 # ip link set br27 mtu 9000 # ip link show br27 8: br27: mtu 9000 qdisc noqueue state UNKNOWN link/ether 00:25:90:60:ae:aa brd ff:ff:ff:ff:ff:ff
While this behaviour isn’t an unreasonable “safety restriction”, it’s rather non-intuitive to someone familiar with layer 2 networking equipment. Typically, the expectation is that every device on the same L2 segment needs to have it’s MTU manually adjusted and this setting is not transparently coupled with other devices. The confusion is further compounded by a not only unhelpful but actively misleading error messages from both the ip
and ifconfig
utils.