Okay, here's the guide I found that worked after two days of trial and error. Ubuntu would do well by their community to post some instructions for local users. https://www.agiliq.com/blog/2012/07/using-ubuntu-cloud-images-in-kvm/
I was able to find the "vmlinuz-virtual-generic" version kernel in a buried "unpacked" folder in the cloud images releases link from above. https://cloud-images.ubuntu.com/focal/current/unpacked/. The main page has other versions than for Focal https://cloud-images.ubuntu.com. This is the most important part because the initrd-userve.img wont recognize the root disk. You can also use a normal ubuntu kernel.
The gist of it is to use qemu-img create -b ubuntu-*-server-cloudimg-amd64.img -f qcow2 samename.img.qcow2 command to create the VM (or just use the GUI create in virt-mgr). It will give the option to configure before install and then start the VM. You'll want to add the kernel and initrd from the "unpacked" downloads to the Direct Boot section under Boot Options plus these boot args: ro init=/usr/lib/cloud-init/uncloud-init root=/dev/vda ds=nocloud ubuntu-pass=initialpassword. You have to boot twice for the password to work. Actually, just rw root=/dev/vda1 init=/bin/bash will work to create a new user and password.
Notice these instructions say vda. I had to use vda1 because there are two boot partitions vda14 for GPT/bios_boot and vda15 for EFI/ESP/boot-flagged partition. The original raw .img file uses sdaX, fyi.. converting to qcow2 or just using qemu-img is what changes that apparently.
A couple more details:
Older help pages say the kernel files and floppy disk are bundled in the tar file. This isn't the case anymore apparently and led me to more trial and error. The common errors are hanging on "GRUB_FORCE_PARTUUID attempting initrdless boot", "Error unknown command hwmatch" and "random: fast init done". Actually, the first two are when not using Direct Boot. The Ova import in VirtualBox does boot on it's own, so it's figuring out the internal kernel files somehow. I would love to know how to do that in KVM. I looked at the XML files but didnt see an obvious clue beside it setting up a floppy disk, putting me on the search to find the floppy which led me to the main guide page above. The third error I quoted is when these internal kernel files are not found, even though the fstab has correct lables for the boot drives. It doesnt even get to grub, so I cant specify disk UUIDs there, though they match in the grub file (obviously because using Direct Boot doesnt change any of that).
Here is an answer to my original question. Again based on the older tar contents with floppy, etc. It did lead me to looking deeper for the "vmlinuz-virtual" files: Where is the usage of Ubuntu cloud guest image on OpenStack documented?
Other resources:
The kernel args uncloud-init tell cloud-init it is a local VM and the nocloud arg sets up local networking (instead of having to go in and change /etc/netplan/*yaml to remove the fixed mac address, and use netplan apply and then systemctl restart networking)
Here is some more info I've learned:
Trying to set the cloud-init yaml files in an easy way with a local file led me to this solution for using cloud-init with an Ubuntu server iso: https://discourse.ubuntu.com/t/automated-server-install-quickstart/16614. It has a few simple commands to create the files and then runs a neat little one-liner server to create the seedfrom net location. It installs and boots with Kvm. Unfortunately after using the raw image to create a VM in Virtmanager, it wouldnt boot because it couldnt find the /stillmounted/casper/initrd for some reason. So I copied the kernel files out of the mounted iso and tried using them (because I suspect the cloud image above doesnt have audio drivers in its kernel), but it also hung on the casper/initrd files.
So I went back to the VirtualBox ova. The my-seed.iso worked to change the server name. After a long search on how to pass kernel parameters (you cant beside a couple using vboxmanage setextradata), I rediscovered you can press shift while booting to edit the Grub args. After using init=/bin/bash to finally set the password and vga=792 to increase the display size, I found a bug in the image in the visudo file that the #include.. is missing a space which kills sudo. After getting apt to update, I found there is a VirtualBox bug that doesnt allow bidirectional clipboard in recent versions: https://superuser.com/questions/1293042/bidirectional-clipboard-not-working-in-virtualbox. But at least I finally have a way to create a basic Ubuntu VM without having to go through the whole install process. This method should work for many other distros. (Oh also, the touch /etc/cloud/cloud-init.disabled command above seems to work).
Followup tidbits:
- The my-config.iso only works the on the first run. (unless there is a way to trigger cloud-init).
- Hold shift to get into Grub. Use the kernel arg
vga=792 for a larger terminal. Set grub timeout in /etc/default/grub and run update-grub.
- Audio doesnt work on these cloud images without installing
apt install linux-generic (about 800Mbs). And set audio output enabled in Vbox.
Last Update: I finally figured out the key to getting the cloud-image to boot without using the Direct Boot files. Basically, either install the non-kvm, generic kernel version using apt install linux-image-$(uname -r) and change kvm to generic (~90Mb), or use the kvm kernel and ssh into the VM using systemctl enable [email protected] && systemctl start [email protected], then virtsh console <vm-name>. To get in initially to make a user use root=/dev/vda1 init=/bin/bash rw for the kernel args. Then delete or move two files /etc/grub.d/01.. and /etc/default/grub.d/50.., and comment out the first line of /etc/default/grub.d/40-force-partuuid.cfg. Change /etc/default/grub hidden to menu, adjust timeout, etc if desired. Then update-grub and it should work.
