qemu-guest-agent fails at autoinstall

This is my first post here so please have patience :)

I am trying to build a Ubuntu 20.04.5 image with Packer(1.8.4) on Proxmox(7.2-11). Everything seems to be working fine (get IP, reads cloud-init config via HTTP, starts the install, installs kernel) until the installation of qemu-guest-agent with subiquity. It fails to run the install command, generates a crash report and asks to hit enter to get a terminal. For 20.04.4 ISO image everything works fine with the same exactly config in Packer.

cloud-init config:

#cloud-config
autoinstall:
  version: 1
  locale: en_US
  keyboard:
    layout: en
  network:
    version: 2
    ethernets:
      ens18:
        dhcp4: true
  ssh:
    install-server: true
    allow-pw: false
    disable_root: true
    ssh_quiet_keygen: true
    allow_public_ssh_keys: true
  packages:
    - qemu-guest-agent
    - sudo
  storage:
    swap:
      size: 0
    config:
      - {ptable: gpt, path: /dev/vda, preserve: false, name: '', grub_device: true, type: disk, id: disk-vda}
      - {type: partition, number: 1, device: disk-vda, flag: bios_grub, size: 1M, id: vda-grub}
      - {type: partition, number: 2, device: disk-vda, flag: boot, size: 1G, id: vda-boot}
      - {type: partition, number: 3, device: disk-vda, size: -1, id: vda-lvm}
      - {type: lvm_volgroup, name: vg-ubuntu, devices: [vda-lvm], id: vg-ubuntu}
      - {type: lvm_partition, volgroup: vg-ubuntu, id: lv-root, name: lv-root, size: -1}
      - {type: format, fstype: ext4, volume: vda-boot, id: vda-boot-fs}
      - {type: format, fstype: xfs, volume: lv-root, id: lv-root-fs}
      - {type: mount, path: /, id: m-root, device: lv-root-fs}
      - {type: mount, path: /boot, id: m-boot, device: vda-boot-fs}
  user-data:
    package_upgrade: true
    timezone: Europe/Bucharest
    users:
      - name: devops
        groups: [adm, sudo]
        lock-passwd: false
        sudo: ALL=(ALL) NOPASSWD:ALL
        shell: /bin/bash
        # passwd: your-password
        ssh_authorized_keys:
          - MyPublicKey

I have no idea if this comes from Ubuntu new iso or Packer but as the same config works for 20.04.4 I think it comes from something new that was included in last release.

Does anyone have an idea or experienced the same?

Thanks in advance for your answers!

Apparently, I have found a band-aid workaround with your issue...

I'm not sure if it would work for you, or if we might have the same problem with qemu-guest-agent returning Error 100.

This is my workaround solution:

I just added a late-commands directive to my user-data file where it would run apt-get update and install qemu-guest-agent after the fact.

here's a snippet of the user-data:

#cloud-config
autoinstall:
...
  late-commands:
    - curtin in-target -- apt-get update
    - curtin in-target -- apt-get install qemu-guest-agent
...

I have looked deeper into the issue to why it failed. Apparently, removing qemu-guest-agent to the list of packages makes everything work fine. When I try running the supposed command inside the error shell, it mentions that "qemu-guest-agent is not found" when it is trying to install that package. This is why I came with this solution while re-reading the docs for automated server install in ubuntu. According to the documentation with regards to running commands on the target machine, it is written there that you should add curtin in-target --target=/target -- before any command for it to run inside the target system. In this case, I want to update the sources and install qemu-guest-agent, and it worked as intended. I do feel like there could be a better solution to this, like configuring the apt directive and such. Hopefully this workaround would work on your case...

I saw a very similar issue to you in my environment and realized that my issue was that I needed to set my corporate proxy to allow apt/snapd reach the internet.

As per the Autoinstall Reference I needed to add the following to my user-data file:

    autoinstall:
    ...
      proxy: http://<proxy.url>:<port-number>
    ...