Score:0

Server

Proxmox VM will not boot my ISO when using KVM

Martin Braun

7/29/23, 9:14 PM

I've setup Proxmox on a vServer that is virtualized using Xen HVM (according to my Hoster) and want to start by creating a VM with pfSense within my vServer using KVM to route my all my traffic through it with the best possible performance, but for some reason I only get this in the console of my new created VM, despite having the ISO uploaded and within its virtual DVD drive:

This is my hardware configuration of my VM:

These are my VM options:

I downloaded the AMD64 DVD installer ISO from the official pfSense download website (Hotlink)

I extracted the ISO on my Mac and uploaded it to my local storage of Proxmox. I also updated the boot order of my VM to boot from the DVD drive first. I also tried to use the noVNC controls on the left to press ESC in the hope to access the Boot Menu, but nothing happens.

I've done a sha256sum check on the downloaded .iso.gz file and the hash is correct.

I figured out that my ISO will not I figured out that my ISO will boot when I disable KVM on the VM and change the processor to qemu64. Obviously, this is no solution, because performance will be terrible without KVM.

Contents of /var/log/syslog:

Mar 30 04:11:46 system pvedaemon[15783]: start VM 100: UPID:system:00003DA7:002F5C28:6243D882:qmstart:100:root@pam:
Mar 30 04:11:46 system pvedaemon[10598]: <root@pam> starting task UPID:system:00003DA7:002F5C28:6243D882:qmstart:100:root@pam:
Mar 30 04:11:47 system systemd[1]: Started 100.scope.
Mar 30 04:11:47 system systemd-udevd[15793]: Using default interface naming scheme 'v240'.
Mar 30 04:11:47 system systemd-udevd[15793]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Mar 30 04:11:47 system systemd-udevd[15793]: Could not generate persistent MAC address for tap100i0: No such file or directory
Mar 30 04:11:47 system kernel: [31037.520616] device tap100i0 entered promiscuous mode
Mar 30 04:11:47 system kernel: [31037.537967] vmbr0: port 2(tap100i0) entered blocking state
Mar 30 04:11:47 system kernel: [31037.537970] vmbr0: port 2(tap100i0) entered disabled state
Mar 30 04:11:47 system kernel: [31037.538177] vmbr0: port 2(tap100i0) entered blocking state
Mar 30 04:11:47 system kernel: [31037.538179] vmbr0: port 2(tap100i0) entered forwarding state
Mar 30 04:11:47 system systemd-udevd[15804]: Using default interface naming scheme 'v240'.
Mar 30 04:11:47 system systemd-udevd[15804]: link_config: autonegotiation is unset or enabled, the speed and duplex are not writable.
Mar 30 04:11:47 system systemd-udevd[15804]: Could not generate persistent MAC address for tap100i1: No such file or directory
Mar 30 04:11:47 system kernel: [31037.958119] device tap100i1 entered promiscuous mode
Mar 30 04:11:47 system kernel: [31037.974101] vmbr1: port 1(tap100i1) entered blocking state
Mar 30 04:11:47 system kernel: [31037.974104] vmbr1: port 1(tap100i1) entered disabled state
Mar 30 04:11:47 system kernel: [31037.974307] vmbr1: port 1(tap100i1) entered blocking state
Mar 30 04:11:47 system kernel: [31037.974310] vmbr1: port 1(tap100i1) entered forwarding state
Mar 30 04:11:48 system pvedaemon[10598]: <root@pam> end task UPID:system:00003DA7:002F5C28:6243D882:qmstart:100:root@pam: OK
Mar 30 04:11:51 system pvedaemon[10598]: <root@pam> starting task UPID:system:00003E03:002F5DFD:6243D887:vncproxy:100:root@pam:
Mar 30 04:11:51 system pvedaemon[15875]: starting vnc proxy UPID:system:00003E03:002F5DFD:6243D887:vncproxy:100:root@pam:
Mar 30 04:12:00 system systemd[1]: Starting Proxmox VE replication runner...
Mar 30 04:12:01 system systemd[1]: pvesr.service: Succeeded.
Mar 30 04:12:01 system systemd[1]: Started Proxmox VE replication runner.
Mar 30 04:13:00 system systemd[1]: Starting Proxmox VE replication runner...
Mar 30 04:13:01 system systemd[1]: pvesr.service: Succeeded.
Mar 30 04:13:01 system systemd[1]: Started Proxmox VE replication runner.

Stdout of dmesg: https://gist.github.com/martin-braun/7c401af831e35343474e6a6aa69c11cc

I tried to configure and run the VM manually from command line:

$ qm set 100 --bootdisk ide2
update VM 100: -bootdisk ide2
$ qm start 100

No more output, the result is the same.

What can I do?

150

0 + 0

boot

proxmox

kvm-virtualization

drookie

7/30/23, 2:15 AM

You should definitely check the logs. Console could be silent but hypervisor logs won't.

0

Reply

Martin Braun

7/30/23, 4:24 AM

@drookie Thanks for the hint, I attached the syslog. The `systemd-udevd` issue [seems to be harmless](https://forum.proxmox.com/threads/proxmox-ve-5-cluster-some-strange-log.35670/#post-187350). How can I print out logs from the VM directly, `qm start 100` is silent.

0

Reply

Nikita Kipriyanov

7/30/23, 5:40 AM

Does it run any other images? Also, which Proxmox version (generally, the "manager version" from the GUI node summary page)? What is the hardware you're running it on?

0

Reply

Nikita Kipriyanov

7/30/23, 5:47 AM

It is certainly problem with your particular installation. I downloaded an image from your hotlink and it successfully boots on PVE 7.1-11. Also I want to state just for the sake of consistency that OS Type is **not a Linux 2.6-5.x**, but FreeBSD (there is no such variant, so I'd choose "other"). But it boots even if you created a VM with "Linux" OS type. I believe it only affects a default virtual hadrware selections.

0

Reply

drookie

7/30/23, 3:55 PM

I'd expect libvirtd logs to live in separate dir and thus be excluded from the default rsyslog catch-all termination to /var/log/messages or /var/log/syslog. Check the **/var/log/libvirt/qemu** dir.

0

Reply

Martin Braun

7/31/23, 6:31 AM

@NikitaKipriyanov Good morning and thanks for your comments. I tried to boot into a Debian ISO and I get the exact same result, so it has nothing to do with the ISO / pfSense. This is PVE 6.4-14 on a vServer at my host. I have no access to the hardware and limited HVM setup capabilities (I had to pick PVE 6), but I'm root on the host machine. I also tried "Other" as OS Type, but as you assumed, it will make no difference indeed.

0

Reply

Martin Braun

7/31/23, 6:36 AM

@drookie There is no `libvirt` folder in `/var/log`, unfortunately.

0

Reply

Nikita Kipriyanov

7/31/23, 6:51 AM

What is vServer? Are you running PVE inside some virtual machine? For that you have to enable *nested virtualization*. Refer to the base platform documentation on how to do this. Also, why using PVE at all? Why won't you run your pfSense on the base platform instead? // @drookie **Proxmox VE** has nothing to do with `libvirt`. It has much better VM manager, serously.

0

Reply

Martin Braun

7/31/23, 6:58 AM

@NikitaKipriyanov I know, yes, I am virtualizing PVE, but nested virtualization is enabled, otherwise PVE would complain. My Hoster also officially supports this. I wish to virtualize pfSense within PVE to avoid the need of a secondary vServer, although I know that this is less effective, since the host will not be hidden behind pfSense. It's still better than not hosting any Firewall at all. I want to keep it cost effective at this point, until I upgrade later. But whatsoever, no ISO is booting, can I assume a problem with my Hoster at this point? Should I get in contact with them?

0

Reply

Nikita Kipriyanov

7/31/23, 7:05 AM

Yes, contact the hoster. But in the meantime I suggest you to read `dmesg` on the PVE thoroughly, because it may suggest you where the problem may be.

0

Reply

Martin Braun

8/2/23, 6:11 PM

@NikitaKipriyanov I contacted my Hoster, they don't have a solution for me, but they figured that I can boot from the ISO when I disable KVM and switch the processor to qemu64 I can boot from the ISO. My Hoster said "Nested Virtualization is not officially supported", which I'm unsure how it's meant. Afaik Nested Virtualization is not Hardware Virtualization through the Kernel Virtualization Module, I think they mean they do not provide support for cases such as mine, thus I will receive no more help with my issue.

0

Reply

Martin Braun

8/2/23, 6:12 PM

They suggested to upgrade PVE to version 7 which sounds ridiculous to me, since they don't even provide a way to install PVE 7 upfront with their HVM. I edited my question to provide `dmesg` logs as well.

0

Reply

Nikita Kipriyanov

8/11/23, 5:26 PM

I don't see a problem. Upgrade. It's better. Instructions to upgrade are on the PVE site.

0

Reply

Martin Braun

8/12/23, 3:16 PM

@NikitaKipriyanov I wiped everything installed PVE7, installed Certbot and replaced the certificates to be able to reach my Proxmox via HTTPS again, logged in, uploaded my ISO and created my VM (very same settings like previously). I did no other updates or anything else. When I try to boot all it says on the noVNC screen is "Guest has not initialized the display (yet).". It's basically the same issue. What keeps popping up is "Failed to run vncproxy", but I found no fix for that so far.

0

Reply

Martin Braun

8/12/23, 3:23 PM

@NikitaKipriyanov Except for disabling KVM. If I disable KVM it works, but performance will be terrible. Are you sure KVM should work on my vServer when it's virtualized using Xen by my Hoster?

0

Reply

Nikita Kipriyanov

8/12/23, 5:24 PM

Last time I run Xen was 10+ years ago, that was Xen v3.4, and hardware of the time did not support any nested virtualization. I don't know if Xen supports it now. Sorry.

0

Reply

Martin Braun

8/12/23, 6:39 PM

@NikitaKipriyanov Finally my Hoster spoke clear. Their vServers don't support nested virtualization using KVM. I quit them. Thanks for all the help whatsoever. :)

0

Reply

Score:0

Server

Martin Braun

8/12/23, 6:40 PM

Apparently my Hoster is NOT supporting KVM on their vServers, so I have to look for different options or go for a dedicated server.

0 + 0