Score:0

KVM/QEMU Virtualization on server with multiple nics to WAN - full split?

no flag

long time reader, first time poster...it's been over a decade, crazy.

Today, I've set up a left over server from my old job in a local data center for a friend of mine and myself.

As by my contract, the data center colocation includes 2 separate switch ports and a /29 net of public ips.

To split my friend's server stuff from mine entirely and to be able to use the distro of choice, I have set up the server with Ubuntu (22.04) and then used libvirt to create two QEMU-guests on this hosts.

eno1 and eno2 as two of the server's network interfaces are connected to the switch, on top, eno1 is serving as combined NIC for the server's IPMI interface (don't worry, it's not fully exposed to the internet, the IPMI firewall only allows connections from one specific ip address I own).

The goal is to be able to use both NICs to their full potential, so I wanted to

  • use eno1 for the host node itself and IPMI and bridge it as br0 to serve to the VM, ip xx.xx.xx.78, gateway .75 - this works

Since i cannot just use eno2 => br1 for the second vm (for obvious reasons, can only have 1 default route via one device to internet), i came up with the following concept:

  • passthrough eno2 as a macvtap private device to the second vm and have it use that device as its own private connection to the internet, ip xx.xx.xx.89, gateway .75 - this doesn't work. Here's the libvirt config for this device:
<interface type='direct'>
  <mac address='ab:cd:ef:00:11:22'/>
  <source dev='eno2' mode='private'/>
  <model type='virtio'/>
  <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</interface>

After booting the guest, the macvtap device shows up on the host node and is up:

13: macvtap0@eno2: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 500
    link/ether ab:cd:ef:00:11:22 brd ff:ff:ff:ff:ff:ff

Now I am at a bit of a loss - is this even doable? Can I use two physical links connected to the host node for separate default routed internet connections from host node+vm1 through NIC no. 1 and then also from vm2 through NIC no. 2?

btw: Since I only have two network ports at my disposal, I can't run the two nics on the host node as a bond with TLB or similar to balance the load with both vms using the same bridge - this way, i wouldn't be able to keep my IPMI connection without a dedicated port..

Best

Chris

I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.