It looks like the layout option does not work with custom grub settings. The installer code appears to only use the grub configuration when layout is not used. You can try submitting a bug report at https://bugs.launchpad.net/subiquity if you think that is the wrong behavior.
Testing - using Ubuntu 20.04.3 (subiquity 21.08.2).
Using this configuration in user-data
#cloud-config
autoinstall:
...
storage:
layout:
name: direct
grub:
reorder_uefi: False
generated this line in /var/log/installer/subiquity-curtin-install.conf without the reorder_uefi option.
grub: {probe_additional_os: true, terminal: unmodified}
Using this configuration in user-data
#cloud-config
autoinstall:
...
storage:
grub:
reorder_uefi: False
config:
- {ptable: gpt, path: /dev/sda, preserve: false, name: '', grub_device: false,
type: disk, id: disk-sda}
- {device: disk-sda, size: 536870912, wipe: superblock, flag: boot, number: 1,
preserve: false, grub_device: true, type: partition, id: partition-sda1}
- {fstype: fat32, volume: partition-sda1, preserve: false, type: format, id: format-2}
- {device: disk-sda, size: 1073741824, wipe: superblock, flag: linux, number: 2,
preserve: false, grub_device: false, type: partition, id: partition-sda2}
- {fstype: ext4, volume: partition-sda2, preserve: false, type: format, id: format-0}
- {device: disk-sda, size: -1, flag: linux, number: 3, preserve: false,
grub_device: false, type: partition, id: partition-sda3}
- name: vg-0
devices: [partition-sda3]
preserve: false
type: lvm_volgroup
id: lvm-volgroup-vg-0
- {name: lv-root, volgroup: lvm-volgroup-vg-0, size: 100%, preserve: false,
type: lvm_partition, id: lvm-partition-lv-root}
- {fstype: ext4, volume: lvm-partition-lv-root, preserve: false, type: format,
id: format-1}
- {device: format-1, path: /, type: mount, id: mount-2}
- {device: format-0, path: /boot, type: mount, id: mount-1}
- {device: format-2, path: /boot/efi, type: mount, id: mount-3}
generated this line in /var/log/installer/subiquity-curtin-install.conf.
grub: {probe_additional_os: true, reorder_uefi: false, terminal: unmodified}
how it works
As part of the install, the server live-installer (subiquity) will generate a config for curtin(/var/log/installer/subiquity-curtin-install.conf) and run curtin. It is curtin that actually performs the disk partitioning (and many other steps). Part of the curtin config is the storage section. This is very similar to the storage section provided in the autoinstall file, but subiquity adds some additional features beyond what curtin supports. The layout option is one of these features. When layout is used, subiquity programmatically generates the curtin storage config and ignores the other user provided settings.