Can't add vxlan to bond interface

Maya

12/1/23, 11:51 AM

I'm trying to add vxlan100 to bond0 interface but it doesn't seem to be working

Here are the commands. I have a wg0 (wireguard) interface on which I'm creating L2 vxlan100.

sudo ip link add vxlan100 type vxlan id 100 dev wg0 
sudo ip link add bond0 type bond
echo 0 > /sys/class/net/bond0/bonding/mode

sudo ip link set vxlan100 master bond0

However the last line "sudo ip link set vxlan100 master bond0" gives me an error

RTNETLINK answers: Invalid argument

Please help, I'm not sure where I got it wrong.

253

1 + 0

linux-networking

vxlan

Score:0

Server

A.B

12/1/23, 4:45 PM

Reproducing this gets a specific kernel message through dmesg:

bond0: (slave vxlan100): Error -22 calling dev_set_mtu

(and we have EINVAL = 22 = Invalid argument)

So it's about MTU. Indeed WireGuard has MTU overhead, so is typically at MTU 1420, and VXLAN will use the parent interface's MTU minus 50 bytes (20 (IP header) + 8 (UDP header) + 8 (VXLAN specific header) + 14 (MAC header) = 50). 1420-50=1370 MTU. A few tests show that the VXLAN's MTU can't be set higher than its initial value: 1370 (but could be set lower) or would get a EINVAL.

When setting vxlan100 as bond0's slave, kernel attempts to set the slave interface's MTU to its master interface's MTU with its default of 1500 but this fails because of above: vxlan100's MTU can't be increased beyond 1370.

To have the operation succeed, the master's MTU has to be lowered in advance:

ip link set dev bond0 mtu 1370

So below commands (reordered, shortened and corrected to use the IANA assigned port 4789) will work:

ip link add name bond0 mtu 1370 type bond mode balance-rr
ip link add name vxlan100 master bond0 type vxlan id 100 dev wg1 dstport 4789

Note: the information about the maximum MTU doesn't appear to be available with ip -d link show dev vxlan100 where there is maxmtu 65535. I don't know if it's a bug or an other unavailable property.

To avoid having to guess the correct MTU, one can use JSON output and the jq command to copy vxlan100's MTU to bond0:

ip link add name vxlan100 type vxlan id 100 dev wg0 dstport 4789
ip link add name bond0 mtu $(ip -json link show dev vxlan100 | jq -r '.[].mtu') type bond mode balance-rr
ip link set dev vxlan100 master bond0

Of course similar MTU should be used in every other related places (eg: the other slaves of bond0 and the peer link with which bond0 communicates), or other MTU-related problems are to be expected. The other way around to keep a 1500 MTU for bond0: having WireGuard at MTU 1550 and thus with the outer layer with an MTU of 1630 doesn't look easy to achieve.

+ 0

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Can't add vxlan to bond interface

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.