Join Log in
Score:1
avatar Ubuntu

KVM Guest on Ubuntu 22.04 doesn't pass traffic over bridged connection

in flag
LifeBoy

I have a relatively straightforward setup for KVM guests using libvirt on Ubuntu 22.04. (All packages updated today). I want a bridged network so I can reach the guests from other machines on the LAN. However traffic is not passed to or from the guest.

My bridge config:

$ sudo brctl show
bridge name bridge id       STP enabled interfaces
vmbr0       8000.8690c059a7cf   yes     eno1
                                        vnet5

$ sudo nmcli conn show --active
NAME                  UUID                                  TYPE      DEVICE 
VO Bridge Connection  71a4f8dd-4b89-4052-be12-21559df85d7b  bridge    vmbr0  
vnet5                 1efe8a7d-6883-42da-acec-0a6e1c03317b  tun       vnet5  
VO-bridge0            8ca30e55-c339-4a43-b0cc-6627fc045a20  ethernet  eno1

(each time the KVM guest is started the network interface # is incremented by 1, so the above started at vnet0 and is now at vnet5 after 5 restarts.)

$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master vmbr0 state UP mode DEFAULT group default qlen 1000
    link/ether e8:40:f2:ac:ff:31 brd ff:ff:ff:ff:ff:ff
    altname enp0s25
3: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 86:90:c0:59:a7:cf brd ff:ff:ff:ff:ff:ff
9: vnet5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fe:40:f2:ac:ff:31 brd ff:ff:ff:ff:ff:ff

The libvirt network config:

$ virsh net-dumpxml vo-br0
<network connections='1'>
  <name>vo-br0</name>
  <uuid>0db0d301-3e92-4e10-a70a-5f3f583234af</uuid>
  <forward mode='bridge'/>
  <bridge name='vmbr0'/>
</network>

$ virsh net-list --all
 Name     State    Autostart   Persistent
-------------------------------------------
 vo-br0   active   yes         yes

The KVM guest network config:

    <interface type='network'>
      <mac address='e8:40:f2:ac:ff:31'/>
      <source network='vo-br0'/>
      <model type='rtl8139'/>
      <link state='up'/>
      <address type='pci' domain='0x0000' bus='0x10' slot='0x01' function='0x0'/>
    </interface>

$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether e8:40:f2:ac:ff:31 brd ff:ff:ff:ff:ff:ff
    altname enp2s1
    inet 192.168.101.6/24 brd 192.168.101.255 scope global noprefixroute ens1
       valid_lft forever preferred_lft forever

However, I cannot even ping the host on the bridge:

$ ping 192.168.101.11
PING 192.168.101.11 (192.168.101.11) 56(84) bytes of data.
From 192.168.101.6 icmp_seq=1 Destination Host Unreachable
From 192.168.101.6 icmp_seq=2 Destination Host Unreachable
From 192.168.101.6 icmp_seq=3 Destination Host Unreachable
^C

and from the host to the guest:

$ ping 192.168.101.6
PING 192.168.101.6 (192.168.101.6) 56(84) bytes of data.
^C
--- 192.168.101.6 ping statistics ---
6 packets transmitted, 0 received, 100% packet loss, time 5114ms

Here is a dump of the traffic on the guest port part of the bridge from the host:

$ sudo tcpdump -i 3
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on vnet5, link-type EN10MB (Ethernet), snapshot length 262144 bytes
10:17:53.452257 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
10:17:54.331497 STP 802.1d, Config, Flags [none], bridge-id 8000.86:90:c0:59:a7:cf.8002, length 35
10:17:54.476238 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
10:17:55.500338 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
10:17:56.333940 STP 802.1d, Config, Flags [none], bridge-id 8000.86:90:c0:59:a7:cf.8002, length 35
10:17:56.524239 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
10:17:57.548256 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
10:17:58.335624 STP 802.1d, Config, Flags [none], bridge-id 8000.86:90:c0:59:a7:cf.8002, length 35
10:17:58.573947 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28

As can be seen, the APR request gets to the host, but the host doesn't know the MAC address of the guest. At least that's what is looks like to me.

No iptables defined that could be blocking this either:

$ sudo iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Network interfaces are up:

$ sudo tcpdump -D
1.eno1 [Up, Running, Connected]
2.vmbr0 [Up, Running, Connected]
3.vnet5 [Up, Running, Connected]
4.any (Pseudo-device that captures on all interfaces) [Up, Running]
5.lo [Up, Running, Loopback]
6.bluetooth-monitor (Bluetooth Linux Monitor) [Wireless]
7.nflog (Linux netfilter log (NFLOG) interface) [none]
8.nfqueue (Linux netfilter queue (NFQUEUE) interface) [none]
9.dbus-system (D-Bus system bus) [none]
10.dbus-session (D-Bus session bus) [none]

I'm strumped. I previously has docker on this host, but have disabled that. I also have some OpenVPN connection, which I have not activated. Both docket and OpenVPN network fine, but KVM doesn't.

I have not been able to find any indication of what could be causing this. I have searched the web fruitlessly. Does anyone have an idea of what I can do to fix this?

Edit: routes:

$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.101.1   0.0.0.0         UG    425    0        0 vmbr0
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 eno1
192.168.88.0    0.0.0.0         255.255.255.0   U     425    0        0 vmbr0
192.168.101.0   0.0.0.0         255.255.255.0   U     425    0        0 vmbr0

Additional update:

@stack3r suggested that device eno1 should have the gateway (and ip address) rather than the vmbr0 bridge. I changed that to test this, but the result is similar.

$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.101.1   0.0.0.0         UG    100    0        0 eno1
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 vmbr0
192.168.101.0   0.0.0.0         255.255.255.0   U     100    0        0 eno1

$ sudo nmcli conn show --active
[sudo] password for roland: 
NAME                  UUID                                  TYPE      DEVICE 
VO eth bridge         25248883-172d-47af-9d3c-0000e2f0d9af  ethernet  eno1   
vnet0                 c9f2a3c4-ea03-4b8d-9dac-781b0d14ef00  tun       vnet0  
VO Bridge Connection  71a4f8dd-4b89-4052-be12-21559df85d7b  bridge    vmbr0  

$ ping 192.168.101.6
PING 192.168.101.6 (192.168.101.6) 56(84) bytes of data.
From 192.168.101.11 icmp_seq=1 Destination Host Unreachable
From 192.168.101.11 icmp_seq=2 Destination Host Unreachable
From 192.168.101.11 icmp_seq=3 Destination Host Unreachable
^C
--- 192.168.101.6 ping statistics ---
6 packets transmitted, 0 received, +3 errors, 100% packet loss, time 5125ms

$ sudo tcpdump -D
1.eno1 [Up, Running, Connected]
2.vmbr0 [Up, Running, Connected]
3.vnet0 [Up, Running, Connected]
4.any (Pseudo-device that captures on all interfaces) [Up, Running]
5.lo [Up, Running, Loopback]
6.bluetooth-monitor (Bluetooth Linux Monitor) [Wireless]
7.nflog (Linux netfilter log (NFLOG) interface) [none]
8.nfqueue (Linux netfilter queue (NFQUEUE) interface) [none]
9.dbus-system (D-Bus system bus) [none]
10.dbus-session (D-Bus session bus) [none]

$ sudo tcpdump -i 2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on vmbr0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
19:43:20.901703 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
19:43:21.925619 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
19:43:22.949575 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
19:43:23.973649 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
19:43:24.997518 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
19:43:26.021477 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
19:43:27.045600 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
19:43:28.069454 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
19:43:29.093423 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
19:43:30.117488 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
^C

$ sudo tcpdump -i 3
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on vnet0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
19:43:52.340139 STP 802.1d, Config, Flags [none], bridge-id 8000.fe:40:f2:ac:ff:31.8001, length 35
19:43:52.644833 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
19:43:53.668768 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
19:43:54.356125 STP 802.1d, Config, Flags [none], bridge-id 8000.fe:40:f2:ac:ff:31.8001, length 35
19:43:54.692879 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
19:43:55.716722 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
19:43:56.340150 STP 802.1d, Config, Flags [none], bridge-id 8000.fe:40:f2:ac:ff:31.8001, length 35
19:43:56.740703 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
19:43:57.764782 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
19:43:58.356147 STP 802.1d, Config, Flags [none], bridge-id 8000.fe:40:f2:ac:ff:31.8001, length 35
19:43:58.788654 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
19:43:59.812633 ARP, Request who-has 192.168.101.1 tell 192.168.101.6, length 28
19:44:00.340132 STP 802.1d, Config, Flags [none], bridge-id 8000.fe:40:f2:ac:ff:31.8001, length 35
^C
583
1 + 4
stack3r avatar
pk flag
stack3r
Hi @LifeBoy, what is the result of "ip route" on your host? With OVPN and Docker there may be some issue with "default" gateway and maybe your ping is simply going to the wrong interface
0
Reply
in flag
LifeBoy
@stack3r, I'm pretty certain that's not the case, but I'll edit my question to include that. Check the tcpdump I shared. It shows that no-one knows where 192.168.101.1 (the default gateway) is...
0
Reply
stack3r avatar
pk flag
stack3r
Hi @LifeBoy, I am not sure if this is relevant for your case, but in my case, the Destination 0.0.0.0 is with Iface of actual physical network card, whereas yours appears to be looping back to the bridge. Is this intentional?
0
Reply
in flag
LifeBoy
@stack3r, I changed this now, so the bridge doesn't have an ip address and eno1 has one and the default gateway too. I don't get a different result though. The bridge still doesn't pass traffic for the KVM guest.
0
Reply
Score:2
avatar Ubuntu
in flag
LifeBoy

I have got it working. I retraced my steps again, deleted the ethernet interface and removed it from the bridge. I then used nm-connection-editor to add a new ethernet port (for eno1) to the bridge.

I assigned an ip to the bridge. I also removed and added the nic configuration to the KVM guest.

After restarting the machine, the bridge came up and the guest has network connectivity.

$ sudo tcpdump -D
1.eno1 [Up, Running, Connected]
2.vmbr0 [Up, Running, Connected]
3.vnet0 [Up, Running, Connected]
4.tun0 [Up, Running, Connected]
5.tun1 [Up, Running, Connected]
6.any (Pseudo-device that captures on all interfaces) [Up, Running]
7.lo [Up, Running, Loopback]
8.bluetooth-monitor (Bluetooth Linux Monitor) [Wireless]
9.nflog (Linux netfilter log (NFLOG) interface) [none]
10.nfqueue (Linux netfilter queue (NFQUEUE) interface) [none]
11.dbus-system (D-Bus system bus) [none]
12.dbus-session (D-Bus session bus) [none]

tun0 and tun1 were not present before.

It seems something went wrong when I added the bridge and interface to it the first time. On recreating it all, it worked as advertised.

$ ping 192.168.101.6
PING 192.168.101.6 (192.168.101.6) 56(84) bytes of data.
64 bytes from 192.168.101.6: icmp_seq=1 ttl=64 time=0.318 ms
64 bytes from 192.168.101.6: icmp_seq=2 ttl=64 time=0.215 ms
^C
--- 192.168.101.6 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1020ms

$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.101.1   0.0.0.0         UG    425    0        0 vmbr0
169.254.0.0     0.0.0.0         255.255.0.0     U     1000   0        0 eno1
192.168.101.0   0.0.0.0         255.255.255.0   U     425    0        0 vmbr0
+ 0
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.