Score:1

Forcing static IP on libvirt

cn flag

I've seen this question and others online, yet my problem persists.

I have a gentoo host with the following interface:

virbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 52:54:00:c0:12:c5  txqueuelen 1000  (Ethernet)
        RX packets 22  bytes 2632 (2.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1880  bytes 99816 (97.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

I have a devuan VM and i'd like it to use a static IP. This is the configuration of the default network in libvirt:

# virsh net-dumpxml default
<network>
  <name>default</name>
  <uuid>469df392-3b72-4069-814a-10893732b627</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:c0:12:c5'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'>
        <lease expiry='0'/>
      </range>
      <host mac='52:54:00:67:37:7d' name='devuan' ip='192.168.122.190'>
        <lease expiry='0'/>
      </host>
    </dhcp>
  </ip>
</network>

With the VM off, i have these leases:

# virsh net-dhcp-leases default
 Expiry Time           MAC address         Protocol   IP address           Hostname   Client ID or DUID
---------------------------------------------
 1970-01-01 01:00:00   52:54:00:67:37:7d   ipv4       192.168.122.190/24   devuan     ff:00:67:37:7d:00:01:00:01:2b:77:a8:75:52:54:00:67:37:7d

...but as soon as i start the VM, the IP gets bumped:

# virsh net-dhcp-leases default
 Expiry Time           MAC address         Protocol   IP address           Hostname   Client ID or DUID
---------------------------------------------
 1970-01-01 01:00:00   52:54:00:67:37:7d   ipv4       192.168.122.191/24   devuan     ff:00:67:37:7d:00:01:00:01:2b:77:ba:24:52:54:00:67:37:7d
 1970-01-01 01:00:00   52:54:00:67:37:7d   ipv4       192.168.122.192/24   -          ff:00:67:37:7d:00:01:00:01:2b:77:b9:7a:52:54:00:67:37:7d

I've tried destroying/starting the network, re/starting libvirtd and virtlogd, kill'ing dnsmasq-related processes in between and even editing /var/lib/libvirt/dnsmasq/virbr0.status to have just the one IP i want. It never pins.

At the moment i also have:

# cat /var/lib/libvirt/dnsmasq/virbr0.macs
[
  {
    "domain": "devuan",
    "macs": [
      "52:54:00:67:37:7d"
    ]
  }
]

# cat /var/lib/libvirt/dnsmasq/virbr0.status
[
  {
    "ip-address": "192.168.122.192",
    "mac-address": "52:54:00:67:37:7d",
    "hostname": "devuan",
    "client-id": "ff:00:67:37:7d:00:01:00:01:2b:77:ba:24:52:54:00:67:37:7d",
    "expiry-time": 0
  },
  {
    "ip-address": "192.168.122.191",
    "mac-address": "52:54:00:67:37:7d",
    "client-id": "ff:00:67:37:7d:00:01:00:01:2b:77:b9:7a:52:54:00:67:37:7d",
    "expiry-time": 0
  }
]

There are no leases on the host's router for this network, so it's not some bleedthrough issue. The VM is set to DHCP. Every time i restart it, it gets a different IP.

/var/lib/libvirt/dnsmasq/default.addnhosts is empty.

# cat /var/lib/libvirt/dnsmasq/default.conf
##WARNING:  THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
##OVERWRITTEN AND LOST.  Changes to this configuration should be made using:
##    virsh net-edit default
## or other application using the libvirt API.
##
## dnsmasq conf file created by libvirt
strict-order
pid-file=/run/libvirt/network/default.pid
except-interface=lo
bind-dynamic
interface=virbr0
dhcp-range=192.168.122.2,192.168.122.254,255.255.255.0,infinite
dhcp-no-override
dhcp-authoritative
dhcp-lease-max=253
dhcp-hostsfile=/var/lib/libvirt/dnsmasq/default.hostsfile
addn-hosts=/var/lib/libvirt/dnsmasq/default.addnhosts

# cat /var/lib/libvirt/dnsmasq/default.hostsfile
52:54:00:67:37:7d,192.168.122.190,devuan,infinite

The interface is as such in the VM's XML configuration:

<interface type='network'>
  <mac address='52:54:00:67:37:7d'/>
  <source network='default' portid='3a1d7ba6-7646-4af0-a28d-36ef38413d30' bridge='virbr0'/>
  <target dev='vnet1'/>
  <model type='virtio'/>
  <alias name='net0'/>
  <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
</interface>

What am i missing?


Edit 1

After @larsks comment i looked at the guest and logs and can trace the following.

root@devuan:~# rm /var/lib/dhcp/* && poweroff

Then at the host:

host ~ # virsh net-destroy default
host ~ # rm /var/lib/libvirt/dnsmasq/virbr0.*

Feb 14 22:45:01 host dnsmasq[24023]: started, version 2.86 cachesize 150
Feb 14 22:45:01 host dnsmasq[24023]: compile time options: IPv6 GNU-getopt no-DBus no-UBus i18n no-IDN DHCP DHCPv6 no-Lua no-TFTP no-conntrack ipset no-auth no-cryptohash no-DNSSEC no-ID loop-detect inotify dumpfile
Feb 14 22:45:01 host dnsmasq-dhcp[24023]: DHCP, IP range 192.168.122.2 -- 192.168.122.254, lease time infinite
Feb 14 22:45:01 host dnsmasq-dhcp[24023]: DHCP, sockets bound exclusively to interface virbr0
Feb 14 22:45:01 host dnsmasq[24023]: reading /etc/resolv.conf
Feb 14 22:45:01 host dnsmasq[24023]: using nameserver 10.0.0.1#53
Feb 14 22:45:01 host dnsmasq[24023]: read /etc/hosts - 2 addresses
Feb 14 22:45:01 host dnsmasq[24023]: read /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
Feb 14 22:45:01 host dnsmasq-dhcp[24023]: read /var/lib/libvirt/dnsmasq/default.hostsfile

host ~ # virsh start devuan

Feb 14 22:47:52 host kernel: virbr0: port 1(vnet7) entered blocking state
Feb 14 22:47:52 host kernel: virbr0: port 1(vnet7) entered disabled state
Feb 14 22:47:52 host kernel: device vnet7 entered promiscuous mode
Feb 14 22:47:52 host kernel: virbr0: port 1(vnet7) entered blocking state
Feb 14 22:47:52 host kernel: virbr0: port 1(vnet7) entered listening state
Feb 14 22:47:54 host kernel: virbr0: port 1(vnet7) entered learning state
Feb 14 22:47:56 host kernel: virbr0: port 1(vnet7) entered forwarding state
Feb 14 22:47:56 host kernel: virbr0: topology change detected, propagating
Feb 14 22:48:03 host dnsmasq-dhcp[24023]: DHCPDISCOVER(virbr0) 52:54:00:67:37:7d
Feb 14 22:48:03 host dnsmasq-dhcp[24023]: DHCPOFFER(virbr0) 192.168.122.190 52:54:00:67:37:7d
Feb 14 22:48:03 host dnsmasq-dhcp[24023]: DHCPREQUEST(virbr0) 192.168.122.190 52:54:00:67:37:7d
Feb 14 22:48:03 host dnsmasq-dhcp[24023]: DHCPACK(virbr0) 192.168.122.190 52:54:00:67:37:7d devuan
Feb 14 22:48:03 host dnsmasq-dhcp[24023]: DHCPDECLINE(virbr0) 192.168.122.190 52:54:00:67:37:7d
Feb 14 22:48:03 host dnsmasq-dhcp[24023]: disabling DHCP static address 192.168.122.190 for 10m
Feb 14 22:48:13 host dnsmasq-dhcp[24023]: not using configured address 192.168.122.190 because it was previously declined
Feb 14 22:48:16 host dnsmasq-dhcp[24023]: DHCPDISCOVER(virbr0) 52:54:00:67:37:7d
Feb 14 22:48:16 host dnsmasq-dhcp[24023]: DHCPOFFER(virbr0) 192.168.122.191 52:54:00:67:37:7d
Feb 14 22:48:16 host dnsmasq-dhcp[24023]: DHCPREQUEST(virbr0) 192.168.122.191 52:54:00:67:37:7d
Feb 14 22:48:16 host dnsmasq-dhcp[24023]: DHCPACK(virbr0) 192.168.122.191 52:54:00:67:37:7d devuan

host ~ # virsh net-dhcp-leases default
 Expiry Time           MAC address         Protocol   IP address           Hostname   Client ID or DUID
---------------------------------------------
 1970-01-01 01:00:00   52:54:00:67:37:7d   ipv4       192.168.122.191/24   devuan     ff:00:67:37:7d:00:01:00:01:2b:7e:c9:a3:52:54:00:67:37:7d

And back at the guest:

root@devuan:~# ls -l /var/lib/dhcp/dhclient.eth0.leases
-rw-r--r-- 1 root root 457 Feb 14 22:48 /var/lib/dhcp/dhclient.eth0.leases

/var/log/syslog and /var/log/daemon.log
Feb 14 22:48:13 devuan dhclient[417]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 3
Feb 14 22:48:16 devuan dhclient[417]: DHCPOFFER of 192.168.122.191 from 192.168.122.1
Feb 14 22:48:16 devuan dhclient[417]: DHCPREQUEST for 192.168.122.191 on eth0 to 255.255.255.255 port 67
Feb 14 22:48:16 devuan dhclient[417]: DHCPACK of 192.168.122.191 from 192.168.122.1
Feb 14 22:48:16 devuan dhclient[417]: Timeout too large reducing to: 2147483646 (TIME_MAX - 1)
Feb 14 22:48:16 devuan dhclient[417]: bound to 192.168.122.191 -- renewal in 2147483648 seconds.

root@devuan:~# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
          inet 192.168.122.190  netmask 255.255.255.0  broadcast 192.168.122.255
          inet6 fe80::5054:ff:fe67:377d  prefixlen 64  scopeid 0x20<link>
          ether 52:54:00:67:37:7d  txqueuelen 1000  (Ethernet)
          RX packets 176  bytes 15369 (15.0 KiB)
          RX errors 0  dropped 5  overruns 0  frame 0
          TX packets 55  bytes 9878 (9.6 KiB)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
          
root@devuan:~# cat /var/lib/dhcp/dhclient.eth0.leases
lease {
    interface "eth0";
    fixed-address 192.168.122.191;
    option subnet-mask 255.255.255.0;
    option routers 192.168.122.1;
    option dhcp-lease-time 4294967295;
    option dhcp-message-type 5;
    option domain-name-servers 192.168.122.1;
    option dhcp-server-identifier 192.168.122.1;
    option broadcast-address 192.168.122.255;
    option host-name "devuan";
    renew 1 2091/03/05 02:02:24;
    rebind 1 2142/03/19 10:27:53;
    expire 6 2159/03/24 05:16:31;
}

It seems like libvirt is offering the correct .190 address, but the guest is refusing it? However, that's what's reported by ifconfig at the guest. Then the guest asks for another address: .191 now, which is the one it stores in /var/lib/dhcp/dhclient.eth0.leases. But from the guest's logs, it seems as though libvirt's dhcpd only offers .191.

This is the problem:

Feb 14 22:48:03 host dnsmasq-dhcp[24023]: DHCPDECLINE(virbr0) 192.168.122.190 52:54:00:67:37:7d

but i'm not sure who's refusing the IP address and which cache do i erase it from.


Edit 2

So i tried a different approach, limiting the IP range instead, setting the static IP out of the range:

# virsh net-edit default
  ..
  <range start='192.168.122.2' end='192.168.122.189'>
  ..
  <host mac='52:54:00:67:37:7d' name='devuan' ip='192.168.122.190'>

But the end result was the same:

Feb 16 22:42:29 host dnsmasq-dhcp[6100]: DHCPDISCOVER(virbr0) 192.168.122.191 52:54:00:67:37:7d
Feb 16 22:42:29 host dnsmasq-dhcp[6100]: DHCPOFFER(virbr0) 192.168.122.190 52:54:00:67:37:7d
Feb 16 22:42:29 host dnsmasq-dhcp[6100]: DHCPREQUEST(virbr0) 192.168.122.190 52:54:00:67:37:7d
Feb 16 22:42:29 host dnsmasq-dhcp[6100]: DHCPACK(virbr0) 192.168.122.190 52:54:00:67:37:7d devuan
Feb 16 22:42:29 host dnsmasq-dhcp[6100]: DHCPDECLINE(virbr0) 192.168.122.190 52:54:00:67:37:7d
Feb 16 22:42:29 host dnsmasq-dhcp[6100]: disabling DHCP static address 192.168.122.190 for 10m
Feb 16 22:42:38 host dnsmasq-dhcp[6100]: not using configured address 192.168.122.190 because it was previously declined
Feb 16 22:42:41 host dnsmasq-dhcp[6100]: DHCPDISCOVER(virbr0) 192.168.122.191 52:54:00:67:37:7d
Feb 16 22:42:41 host dnsmasq-dhcp[6100]: DHCPOFFER(virbr0) 192.168.122.153 52:54:00:67:37:7d
Feb 16 22:42:41 host dnsmasq-dhcp[6100]: DHCPREQUEST(virbr0) 192.168.122.153 52:54:00:67:37:7d
Feb 16 22:42:41 host dnsmasq-dhcp[6100]: DHCPACK(virbr0) 192.168.122.153 52:54:00:67:37:7d devuan

And at the guest:

Feb 16 22:42:38 devuan dhclient[412]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 8
Feb 16 22:42:41 devuan dhclient[412]: DHCPOFFER of 192.168.122.153 from 192.168.122.1
Feb 16 22:42:41 devuan dhclient[412]: DHCPREQUEST for 192.168.122.153 on eth0 to 255.255.255.255 port 67
Feb 16 22:42:41 devuan dhclient[412]: DHCPACK of 192.168.122.153 from 192.168.122.1
Feb 16 22:42:41 devuan dhclient[412]: Timeout too large reducing to: 2147483646 (TIME_MAX - 1)
Feb 16 22:42:41 devuan dhclient[412]: bound to 192.168.122.153 -- renewal in 2147483648 seconds.

Edit 3

Added two more VMs to the pool, as such (changed to 100s but everything else is the same):

<dhcp>
    <range start='192.168.122.2' end='192.168.122.254'/>
    <host mac='52:54:00:67:37:7d' name='devuan' ip='192.168.122.100'>
      <lease expiry='0'/>
    </host>
    <host mac='52:54:00:3e:96:2e' name='slack' ip='192.168.122.101'>
      <lease expiry='0'/>
    </host>
    <host mac='52:54:00:1d:31:fb' name='win10' ip='192.168.122.102'>
      <lease expiry='0'/>
    </host>
</dhcp>

Which resulted in:

# virsh net-dhcp-leases default
 Expiry Time           MAC address         Protocol   IP address           Hostname   Client ID or DUID
---------------------------------------------
 1970-01-01 01:00:00   52:54:00:1d:31:fb   ipv4       192.168.122.102/24   win10      01:52:54:00:1d:31:fb
 1970-01-01 01:00:00   52:54:00:3e:96:2e   ipv4       192.168.122.101/24   slack      ff:00:3e:96:2e:00:04:82:83:7d:54:99:e4:4f:15:9f:85:b6:f1:80:0e:54:f5
 1970-01-01 01:00:00   52:54:00:67:37:7d   ipv4       192.168.122.191/24   devuan     ff:00:67:37:7d:00:01:00:01:2b:82:9c:6b:52:54:00:67:37:7d

Both Slackware and Windows get their IPs correctly (after shutting down all VMs and restarting the default interface). Devuan insists on .191, although ifconfig reports .100:

host $ ssh 192.168.122.191
devuan $ /sbin/ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
          inet 192.168.122.100  netmask 255.255.255.0  broadcast 192.168.122.255
          inet6 fe80::5054:ff:fe67:377d  prefixlen 64  scopeid 0x20<link>
          ether 52:54:00:67:37:7d  txqueuelen 1000  (Ethernet)
          RX packets 225  bytes 23965 (23.4 KiB)
          RX errors 0  dropped 5  overruns 0  frame 0
          TX packets 61  bytes 12083 (11.7 KiB)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
devuan # cat /var/log/syslog
Feb 17 20:13:41 devuan dhclient[416]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 6
Feb 17 20:13:44 devuan dhclient[416]: DHCPOFFER of 192.168.122.191 from 192.168.122.1
Feb 17 20:13:44 devuan dhclient[416]: DHCPREQUEST for 192.168.122.191 on eth0 to 255.255.255.255 port 67
Feb 17 20:13:44 devuan dhclient[416]: DHCPACK of 192.168.122.191 from 192.168.122.1
Feb 17 20:13:44 devuan dhclient[416]: Timeout too large reducing to: 2147483646 (TIME_MAX - 1)
Feb 17 20:13:44 devuan dhclient[416]: bound to 192.168.122.191 -- renewal in 2147483648 seconds.
devuan # netstat -puta --numeric-hosts --numeric-ports | grep \:22
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1195/sshd: /usr/sbi
tcp        0      0 192.168.122.191:22      192.168.122.1:36434     ESTABLISHED 1255/sshd: joe
tcp6       0      0 :::22                   :::*                    LISTEN      1195/sshd: /usr/sbi

From Devuan's perspective it seems like the only IP that it's getting is .191, but ifconfig says otherwise. From the host's perspective:

Feb 17 20:13:31 host dnsmasq-dhcp[8336]: DHCPDISCOVER(virbr0) 192.168.122.245 52:54:00:1d:31:fb
Feb 17 20:13:31 host dnsmasq-dhcp[8336]: DHCPOFFER(virbr0) 192.168.122.102 52:54:00:1d:31:fb
Feb 17 20:13:31 host dnsmasq-dhcp[8336]: DHCPREQUEST(virbr0) 192.168.122.102 52:54:00:1d:31:fb
Feb 17 20:13:31 host dnsmasq-dhcp[8336]: DHCPACK(virbr0) 192.168.122.102 52:54:00:1d:31:fb win10
Feb 17 20:13:31 host dnsmasq-dhcp[8336]: DHCPDISCOVER(virbr0) 52:54:00:67:37:7d
Feb 17 20:13:31 host dnsmasq-dhcp[8336]: DHCPOFFER(virbr0) 192.168.122.100 52:54:00:67:37:7d
Feb 17 20:13:31 host dnsmasq-dhcp[8336]: DHCPREQUEST(virbr0) 192.168.122.100 52:54:00:67:37:7d
Feb 17 20:13:31 host dnsmasq-dhcp[8336]: DHCPACK(virbr0) 192.168.122.100 52:54:00:67:37:7d devuan
Feb 17 20:13:31 host dnsmasq-dhcp[8336]: DHCPDECLINE(virbr0) 192.168.122.100 52:54:00:67:37:7d
Feb 17 20:13:31 host dnsmasq-dhcp[8336]: disabling DHCP static address 192.168.122.100 for 10m
Feb 17 20:13:41 host dnsmasq-dhcp[8336]: not using configured address 192.168.122.100 because it was previously declined
Feb 17 20:13:45 host dnsmasq-dhcp[8336]: DHCPDISCOVER(virbr0) 52:54:00:67:37:7d
Feb 17 20:13:45 host dnsmasq-dhcp[8336]: DHCPOFFER(virbr0) 192.168.122.191 52:54:00:67:37:7d
Feb 17 20:13:45 host dnsmasq-dhcp[8336]: DHCPREQUEST(virbr0) 192.168.122.191 52:54:00:67:37:7d
Feb 17 20:13:45 host dnsmasq-dhcp[8336]: DHCPACK(virbr0) 192.168.122.191 52:54:00:67:37:7d devuan
Feb 17 20:13:45 host dnsmasq-dhcp[8336]: DHCPDISCOVER(virbr0) 52:54:00:3e:96:2e
Feb 17 20:13:45 host dnsmasq-dhcp[8336]: DHCPOFFER(virbr0) 192.168.122.101 52:54:00:3e:96:2e
Feb 17 20:13:45 host dnsmasq-dhcp[8336]: DHCPREQUEST(virbr0) 192.168.122.101 52:54:00:3e:96:2e
Feb 17 20:13:45 host dnsmasq-dhcp[8336]: DHCPACK(virbr0) 192.168.122.101 52:54:00:3e:96:2e slack

I'm now leaning towards Devuan. The long MAC address issue didn't help.

pt flag
I can't reproduce this behavior: if I create a static lease in the network XML and then restart (`virsh net-destroy default`/`virsh net-start default`) the network, virtual machines acquire the ip address assigned to their MAC. On my system (Fedora 37), the libvirt-managed dnsmasq server logs to the system journal. Do you see any useful information in your logs related to the DHCP requests? Do you see the same behavior if you boot something other than devuan?
vesperto avatar
cn flag
@larsks either the host is refusing to offer or the guest is refusing to accept, i've edited the response, i'm not that well versed on DHCP.
Score:0
cn flag

In the end this solved it for me.

devuan# /etc/init.d/network restart # caused .100 to be assigned
devuan# rm /var/lib/dhcp/dhclient.eth0.leases
devuan# poweroff

Ensured any /var/lib/libvirt/dnsmasq/ files had .100 in it.

host# virsh net-destroy default && virsh net-start default
host# ssh 192.168.122.100
devuan# ifconfig
  eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500    
  inet 192.168.122.100  netmask 255.255.255.0  broadcast 192.168.122.255
devuan# reboot

...still has .100. The /etc/init.d/network restart did it, although i don't know why.

I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.