Mounting Error via fstab

I'm having an issue mounting via fstab which causes my ubuntu server to boot in emergency mode. and I'm not sure what I did, but I did have this working at some point. This is how the code is written in the fstab

/dev/disk/by-id/ata-WDC_WD40EFZX-68AWUNO_WD-WX32DC02K0DD-part1 /mnt/disk1   xfs defaults 0 0
/dev/disk/by-id/ata-WDC_WD40EFZX-68AWUNO_WD-WX32DC0HY19Z-part1 /mnt/disk2   xfs defaults 0 0
/dev/disk/by-id/ata-WDC_WD40EFRX-68N32NO_WD-WCC7K5TUY2LZ-part1 /mnt/disk3   xfs defaults 0 0
/dev/disk/by-id/ata-WDC_WD80EFZZ-68BTXNO_WD-CA28AU9K-part1     /mnt/disk4   xfs defaults 0 0

when I try mounting that via mount -a I get an error that says

> /dev/disk...0DD-part1" Can't open blockdev 
> mount: /mnt/disk1: special device /dev/disk...0DD-part1 does not exist.

This error shows for every disk I try to mount. However, I can mount them fine if I just run the following code from terminal

mount /dev/disk/by-id/ata-WDC_WD40EFZX-68AWUN0_WD-WX32DC02K0DD-part1 /mnt/disk1
mount /dev/disk/by-id/ata-WDC_WD40EFZX-68AWUN0_WD-WX32DC0HY19Z-part1 /mnt/disk2
mount /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K5TUY2LZ-part1 /mnt/disk3
mount /dev/disk/by-id/ata-WDC_WD80EFZZ-68BTXN0_WD-CA28AU9K-part1 /mnt/disk4

Any idea what I'm doing wrong? I have looked up different answers on the forum, but really not helping me get closer to resolving this. I havent recently repartitioned any drives. If I do the manual mounts via terminal, then comment out the fstab lines, I can get my server up and running, but that ultimately causes other problems.

This all seemed to happen after updates and reboot. Not saying that caused it, but I have been having trouble since.

Other possible relevant info Ubuntu 22.04.2 LTS

I did make sure to mkdir /mnt/disk1 - /mnt/disk4. All the drives work as expected if I manually mount them via terminal.

output of cat /etc/fstab

# / was on dev/ubuntu-vg/ubuntu-lv during curtin installation
/dev/disk/by-id/dm-uuid-LVM-UTmmKp12MPQMJOl1dVrDvQKEyta1fUcahtQHdcb1J58rmxz4AmnY5ywRHzso8ivW / ext defaults 01
# /boot was on /dev/sdc2 during curtin installation
/dev/disk/by-uuid/ae9ea8d3-134a-4438-a352-68cd301b666a /boot ext4 defaults 0 1
/swap.img none swap sw 0 0

output of ls /dev/mapper

control  ubuntu--vg-ubuntu--lv

The /dev/disk directory is filled by "udev". I am not sure and could not proof in short time, that systemd starts udev after it handles /etc/fstab.

This means udev did not create the links in /dev/disk at the time the system works on /etc/fstab.

Better use UUID in /etc/fstab instead of the /dev/disk/... path.

Get the UUID with the command blkid, e.g.

blkid /dev/disk/by-id/ata-WDC_WD40EFZX-68AWUN0_WD-WX32DC02K0DD-part1

it will show somthing like

/dev/disk/by-id/ata-WDC_WD40EFZX-68AWUN0_WD-WX32DC02K0DD-part1: ... UUID="Q6MOds-VJGb-L7fd-qZxN-VHoM-PxRz-IDA4hW" ...

now edit /etc/fstab and replace the /dev/disk path with the UUID:

UUID="Q6MOds-VJGb-L7fd-qZxN-VHoM-PxRz-IDA4hW" /mnt/disk1 xfs defaults 0 0

do this with all the other partitions.