Score:0

libvirt with qemu guest, bridged networking does not work

nl flag

I am trying to get a VM operational and working so my router can forward a high port to its SSH, allowing someone on the Internet to connect to the VM. I am familiar with how to make that part of it operational.

I first tried bridged networking. With that, the guest does not get an IP address. If I set an IP address on the guest manually, then the guest can ping Internet addresses, but TCP connections do not work. That is very weird.

If I set it to NAT networking instead, then it gets an IP address and networking works, but I am then unable to get qemu port forwarding to work. Interesting side note: NAT used to assign an IP address to the guest from the virbr0 network range on the containing host (192.168.122.0/24), now it assigns an address from an entirely different network (10.0.2.0/24) which I do not see anywhere on the libvirt host. But I can make successful outbound gonnections from the VM, so it's a curiosity, not a real problem.

I would prefer bridged networking, but if I can get port forwarding working with NAT, that is an acceptable solution.

The libvirt host is Ubuntu Server 22.0.4 and has NIC eno1 included in br0, with the IP address on br0. When I set the guest to bridged networking, I connect it to br0. The bridge on that host seems to work perfectly outside of libvirt. It is running libvirt version 8.0.0-1ubuntu7.2 and qemu version 1:6.2+dfsg-2ubuntu6.4.

The guest is Ubuntu Server 20.04.

EDIT: bridges layouts added from comment:

elyograg@smeagol:~$ brctl show
bridge name     bridge id           STP enabled     interfaces
br0             8000.168b32b94b55   yes             eno1
                                                    eno2
                                                    vnet0 
docker0         8000.0242aa9c3cdf   no              
virbr0          8000.525400728c3a   yes             
nl flag
`elyograg@smeagol:~$ brctl show` `bridge name bridge id STP enabled interfaces` `br0 8000.168b32b94b55 yes eno1` ` eno2` ` vnet0` `docker0 8000.0242aa9c3cdf no` `virbr0 8000.525400728c3a yes`
nl flag
I can't get the `brctl show` output to format nicely. br0 has spanning tree enabled and includes eno1, eno2, and vnet0 which is the guest's bridged nic. eno1 is plugged in, eno2 is not. If I can get another switch installed, I will plug eno2 into it.
A.B avatar
cl flag
A.B
You wrote in comments the important part that was missing in the question itself: `docker0` => Docker is in use.
A.B avatar
cl flag
A.B
Does this answer your question? [Docker breaks libvirt bridge network](https://serverfault.com/questions/963759/docker-breaks-libvirt-bridge-network)
djdomi avatar
za flag
do we speak about a home network? and also take a look [here](https://serverfault.com/questions/170079/forwarding-ports-to-guests-in-libvirt-kvm)
nl flag
I do have docker installed, but it has been installed far longer than libvirt, and I only have "miigotu/certbot-dns-godaddy" and "ymuski/curl-http3" images on it. This server is in my basement, so it's a home network. I figured out the problem.
Score:0
nl flag

This was fixed by adding this line to the sysctl config and using sysctl -w to change it imemediately:

net.bridge.bridge-nf-call-iptables = 0

But then I discovered that using a bridged network was breaking ucarp, which I need. So I got rid of br0, and figured out how to use a libvirt hook to forward a host port to port 22 on the VM in NAT mode, which was the whole reason I wanted the VM to use bridged networking -- making it accessible outside the libvirt host.

Now I am using a team0 interface, which didn't immediately work with ucarp either. Adding ifup team0:ucarp to /etc/rc.local fixed that. When things are calm, I will try br0 again and using ifup br0:ucarp instead.

nl flag
Going back to br0 and doing `ifup br0:ucarp` in /etc/rc.local worked. So now the VM is doing bridged networking and it all works. What a frustrating journey!
nl flag
Looking into what `net.bridge.bridge-nf-call-iptables` actually does, I was worried that it would basically turn off my firewall because it makes it so traffic on the bridge doesn't go through the firewall. That was not a problem, though. It appears that the setting only affects traffic that is traversing the bridge, not traffic sent TO the bridge interface. I started a service listening on the host port 7777 and then tried to connect to it from another host. I did not allow that port with ufw. The traffic was blocked, and logged by syslog.
nl flag
Ultimately I settled on an entirely different solution for the high availability: Pacemaker. It doesn't have the issues with unusual networking configurations that ucarp does. Now the config is quite simple. My real servers have a br0 interface, which allows them to run bridged networking for VMs. VMs have whatever interface the hypervisor gives them. Networking on all Ubuntu hosts, both bare metal and VMs, is configured by netplan, and NetworkManager is not even installed.
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.