amdgpu causing gnome-shell segfault

I have a brand new install of Ubuntu desktop 20.04.3 LTS.

I am getting repeated error of gnome-shell segfault, and vnc is showing me just a black screen.

[Mon Nov  8 22:28:37 2021] gnome-shell[34961]: segfault at 0 ip 00007ff4c9d603dd sp 00007ffcd780d240 error 4 in amdgpu_dri.so[7ff4c88f2000+2938000]
[Mon Nov  8 22:28:37 2021] Code: 04 00 00 48 8b 05 eb 69 11 02 48 89 c7 48 89 44 24 68 e8 d6 84 f2 ff 48 8b 44 24 50 48 8b b8 a8 00 00 00 48 8b b0 90 00 00 00 <48> 8b 07 ff 90 f0 00 00 00 48 8b 0d ab 69 11 02 89 c3 48 85 c9 74
[Mon Nov  8 22:28:38 2021] gnome-shell[35062]: segfault at 0 ip 00007f06a5d603dd sp 00007ffc15d45100 error 4 in amdgpu_dri.so[7f06a48f2000+2938000]
[Mon Nov  8 22:28:38 2021] Code: 04 00 00 48 8b 05 eb 69 11 02 48 89 c7 48 89 44 24 68 e8 d6 84 f2 ff 48 8b 44 24 50 48 8b b8 a8 00 00 00 48 8b b0 90 00 00 00 <48> 8b 07 ff 90 f0 00 00 00 48 8b 0d ab 69 11 02 89 c3 48 85 c9 74
[Mon Nov  8 22:28:39 2021] gnome-shell[35163]: segfault at 0 ip 00007f1b55d603dd sp 00007ffee57e7c10 error 4 in amdgpu_dri.so[7f1b548f2000+2938000]
[Mon Nov  8 22:28:39 2021] Code: 04 00 00 48 8b 05 eb 69 11 02 48 89 c7 48 89 44 24 68 e8 d6 84 f2 ff 48 8b 44 24 50 48 8b b8 a8 00 00 00 48 8b b0 90 00 00 00 <48> 8b 07 ff 90 f0 00 00 00 48 8b 0d ab 69 11 02 89 c3 48 85 c9 74
[Mon Nov  8 22:28:40 2021] gnome-shell[35264]: segfault at 0 ip 00007f9fbdd603dd sp 00007ffd772af970 error 4 in amdgpu_dri.so[7f9fbc8f2000+2938000]
[Mon Nov  8 22:28:40 2021] Code: 04 00 00 48 8b 05 eb 69 11 02 48 89 c7 48 89 44 24 68 e8 d6 84 f2 ff 48 8b 44 24 50 48 8b b8 a8 00 00 00 48 8b b0 90 00 00 00 <48> 8b 07 ff 90 f0 00 00 00 48 8b 0d ab 69 11 02 89 c3 48 85 c9 74
[Mon Nov  8 22:28:41 2021] gnome-shell[35365]: segfault at 0 ip 00007f82bdd603dd sp 00007ffed4d19700 error 4 in amdgpu_dri.so[7f82bc8f2000+2938000]
[Mon Nov  8 22:28:41 2021] Code: 04 00 00 48 8b 05 eb 69 11 02 48 89 c7 48 89 44 24 68 e8 d6 84 f2 ff 48 8b 44 24 50 48 8b b8 a8 00 00 00 48 8b b0 90 00 00 00 <48> 8b 07 ff 90 f0 00 00 00 48 8b 0d ab 69 11 02 89 c3 48 85 c9 74
[Mon Nov  8 22:28:42 2021] gnome-shell[35467]: segfault at 0 ip 00007f7b75d603dd sp 00007ffe062ce350 error 4 in amdgpu_dri.so[7f7b748f2000+2938000]
[Mon Nov  8 22:28:42 2021] Code: 04 00 00 48 8b 05 eb 69 11 02 48 89 c7 48 89 44 24 68 e8 d6 84 f2 ff 48 8b 44 24 50 48 8b b8 a8 00 00 00 48 8b b0 90 00 00 00 <48> 8b 07 ff 90 f0 00 00 00 48 8b 0d ab 69 11 02 89 c3 48 85 c9 74
[Mon Nov  8 22:28:43 2021] gnome-shell[35568]: segfault at 0 ip 00007fc5a5d603dd sp 00007ffc033437e0 error 4 in amdgpu_dri.so[7fc5a48f2000+2938000]
[Mon Nov  8 22:28:43 2021] Code: 04 00 00 48 8b 05 eb 69 11 02 48 89 c7 48 89 44 24 68 e8 d6 84 f2 ff 48 8b 44 24 50 48 8b b8 a8 00 00 00 48 8b b0 90 00 00 00 <48> 8b 07 ff 90 f0 00 00 00 48 8b 0d ab 69 11 02 89 c3 48 85 c9 74
[Mon Nov  8 22:28:45 2021] gnome-shell[35670]: segfault at 0 ip 00007f8fc1d603dd sp 00007ffe8f5c1c10 error 4 in amdgpu_dri.so[7f8fc08f2000+2938000]
[Mon Nov  8 22:28:45 2021] Code: 04 00 00 48 8b 05 eb 69 11 02 48 89 c7 48 89 44 24 68 e8 d6 84 f2 ff 48 8b 44 24 50 48 8b b8 a8 00 00 00 48 8b b0 90 00 00 00 <48> 8b 07 ff 90 f0 00 00 00 48 8b 0d ab 69 11 02 89 c3 48 85 c9 74
[Mon Nov  8 22:28:46 2021] gnome-shell[35772]: segfault at 0 ip 00007ff219d603dd sp 00007ffcdb859d90 error 4 in amdgpu_dri.so[7ff2188f2000+2938000]
[Mon Nov  8 22:28:46 2021] Code: 04 00 00 48 8b 05 eb 69 11 02 48 89 c7 48 89 44 24 68 e8 d6 84 f2 ff 48 8b 44 24 50 48 8b b8 a8 00 00 00 48 8b b0 90 00 00 00 <48> 8b 07 ff 90 f0 00 00 00 48 8b 0d ab 69 11 02 89 c3 48 85 c9 74

I have an AMD Radeon Pro w6600 attached to it, and the latest Radeon Pro drivers installed

I downgraded the kernel to 5.8 as specified in radeon driver instructions here: https://www.amd.com/en/support/kb/release-notes/rn-pro-lin-21-q3 and rebooted before installing the drivers.

This is a VM, running in proxmox, with the graphics card passed through to the VM. As far as I can tell, the settings in proxmox are correct, and the options on the VM are correct.

LSHW:

$ sudo lshw -c video
  *-display
       description: VGA compatible controller
       physical id: 1
       bus info: pci@0000:00:01.0
       version: 02
       width: 32 bits
       clock: 33MHz
       capabilities: vga_controller bus_master rom
       configuration: driver=bochs-drm latency=0
       resources: irq:0 memory:c0000000-c0ffffff memory:c224b000-c224bfff memory:c0000-dffff
  *-display
       description: VGA compatible controller
       product: Navi 23 WKS-XL [Radeon PRO W6600]
       vendor: Advanced Micro Devices, Inc. [AMD/ATI]
       physical id: 10
       bus info: pci@0000:06:10.0
       version: 00
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi vga_controller bus_master cap_list rom
       configuration: driver=amdgpu latency=0
       resources: iomemory:80-7f iomemory:80-7f irq:49 memory:800000000-80fffffff memory:810000000-8101fffff ioport:9000(size=256) memory:c1600000-c16fffff memory:c1720000-c173ffff