Score:0

Kernel-specific failure to mount OS-containing drive

lk flag
ash

I recently received a Dell Precision 7820 with Ubuntu 20.04 pre-installed. After fully updating (and all was working), I upgraded to 22.04, but unfortunately, this has done something to prevent the OS-containing SSD from being visible on boot (but only sometimes...).

On booting via default settings, I'm greeted with:

ALERT! UUID=51139eac-d829-4030-8044-d1328bdfd2d3 does not exist. Dropping to a shell.

After which, I drop into an initramfs shell.

The punchline (which I arrived at after trying many many things) is this is kernel-dependent.

After enabling legacy boot via ROM in the BIOS (which I share only in case this matters) and ensuring that SATA is configured to use ACHI instead of RAID (as discussed in this question) I'm left with the following situation.

On boot and hitting ESC to offer up the Ubuntu GRUB options (I've now edited the GRUB info to show this by default, but including this so someone can reproduce what I did without that):

Ubuntu
Advanced options for Ubuntu
UEFI Firmware Settings 
Restore OS to factory state

If I select Advanced options for Ubuntu and then boot into the 5.19.0-46 kernel (which is the default if I just select Ubuntu from the GRUB list), then this fails to mount and drop into initramfs.

HOWEVER, if I boot into the 5.15.0-76 kernel, Ubuntu starts fine with no issue.

Any help to explain this would be hugely appreciated. I'm seriously concerned that I don't understand why this is happening, which means I'm effectively pinned to 5.15.0-76 because, as of right now later kernels prevent OS loading!

Update in response to oldfred (July 21s)

Boot repair link here

============================== Boot Info Summary ===============================

 => No boot loader is installed in the MBR of /dev/nvme0n1.
 => libparted MBR boot code is installed in the MBR of /dev/sda.

nvme0n1p1: _____________________________________________________________________

    File system:       vfat
    Boot sector type:  Windows 8/10/11/2012: FAT32
    Boot sector info:  No errors found in the Boot Parameter Block.
    Operating System:  
    Boot files:        /efi/Boot/fbx64.efi /efi/Boot/mmx64.efi 
                       /efi/ubuntu/grubx64.efi /efi/ubuntu/loadefi.efi 
                       /efi/ubuntu/mmx64.efi /efi/ubuntu/shimx64.efi 
                       /efi/ubuntu/UbuntuSecBoot.efi /efi/ubuntu/grub.cfg

nvme0n1p2: _____________________________________________________________________

    File system:       vfat
    Boot sector type:  Windows 8/10/11/2012: FAT32
    Boot sector info:  No errors found in the Boot Parameter Block.
    Operating System:  
    Boot files:        /boot/grub/grub.cfg

nvme0n1p3: _____________________________________________________________________

    File system:       ext4
    Boot sector type:  -
    Boot sector info: 
    Operating System:  Ubuntu 22.04.2 LTS
    Boot files:        /boot/grub/grub.cfg /etc/fstab /etc/default/grub

sda1: __________________________________________________________________________

    File system:       ext4
    Boot sector type:  -
    Boot sector info: 
    Operating System:  
    Boot files:        

sda2: __________________________________________________________________________

    File system:       ext4
    Boot sector type:  -
    Boot sector info: 
    Operating System:  
    Boot files:        


================================ 1 OS detected =================================

OS#1:   Ubuntu 22.04.2 LTS on nvme0n1p3

================================ Host/Hardware =================================

CPU architecture: 64-bit
Video: GA104GL [RTX A4000] EFI VGA from NVIDIA Corporation
BOOT_IMAGE of the installed session in use:
/boot/vmlinuz-5.15.0-76-generic root=UUID=51139eac-d829-4030-8044-d1328bdfd2d3 ro quiet splash vt.handoff=7
df -Th / : /dev/nvme0n1p3 ext4  1.9T  179G  1.6T  11% /

===================================== UEFI =====================================

BIOS/UEFI firmware: 2.32.1(2.32) from Dell Inc.
The firmware is EFI-compatible, and is set in EFI-mode for this installed-session.
SecureBoot disabled (confirmed by mokutil).
BootCurrent: 0007
Timeout: 1 seconds
BootOrder: 0007,0029,002A,002B,002C,002D,0000,0025,0026,0027,0028,0023
Boot0000* ubuntu_shimx64    NVMe(0x1,01-00-00-00-00-00-00-00)/HD(1,GPT,cf73badc-18d9-4b5f-afe6-5c279ce0d278,0x800,0x1a9000)/File(\EFI\ubuntu\shimx64.efi)
Boot0001* Diskette Drive    BBS(Floppy,Diskette Drive,0x0)..BO
Boot0002* Internal HDD  BBS(HD,Internal HDD,0x0)..BO
Boot0003* USB Storage Device    BBS(USB,USB Storage Device,0x0)..BO
Boot0004* CD/DVD/CD-RW Drive    BBS(CDROM,CD/DVD/CD-RW Drive,0x0)..BO
Boot0005* Onboard NIC   BBS(Network,IBA CL Slot 00FE v0110,0x0)..BO
Boot0006* load_efi  HD(1,GPT,cf73badc-18d9-4b5f-afe6-5c279ce0d278,0x800,0x1a9000)/File(\EFI\Ubuntu\loadefi.efi)
Boot0007* ubuntu    HD(1,GPT,cf73badc-18d9-4b5f-afe6-5c279ce0d278,0x800,0x1a9000)/File(\EFI\ubuntu\shimx64.efi)
Boot0008* Onboard NIC(IPV4) PciRoot(0x0)/Pci(0x1f,0x6)/MAC(cc96e52d71d5,0)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0009* Onboard NIC(IPV6) PciRoot(0x0)/Pci(0x1f,0x6)/MAC(cc96e52d71d5,0)/IPv6([::]:<->[::]:,0,0)..BO
Boot000A* PXE IP4 Intel(R) Ethernet 10G 2P X710-T2L-t Adapter   PciRoot(0x1)/Pci(0x0,0x0)/Pci(0x0,0x0)/MAC(b48351081138,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot000B* PXE IP6 Intel(R) Ethernet 10G 2P X710-T2L-t Adapter   PciRoot(0x1)/Pci(0x0,0x0)/Pci(0x0,0x0)/MAC(b48351081138,1)/IPv6([::]:<->[::]:,0,0)..BO
Boot000C* PXE IP4 Intel(R) Ethernet Network Adapter X710-TL PciRoot(0x1)/Pci(0x0,0x0)/Pci(0x0,0x1)/MAC(b48351081139,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot000D* PXE IP6 Intel(R) Ethernet Network Adapter X710-TL PciRoot(0x1)/Pci(0x0,0x0)/Pci(0x0,0x1)/MAC(b48351081139,1)/IPv6([::]:<->[::]:,0,0)..BO
Boot0023* UEFI: Hard Drive  HD(1,GPT,cf73badc-18d9-4b5f-afe6-5c279ce0d278,0x800,0x1a9000)/File(EFI\boot\bootx64.efi)..BO
Boot0025  PXE IP4 Intel(R) Ethernet 10G 2P X710-T2L-t Adapter   PciRoot(0x1)/Pci(0x0,0x0)/Pci(0x0,0x0)/MAC(b48351081138,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0026  PXE IP6 Intel(R) Ethernet 10G 2P X710-T2L-t Adapter   PciRoot(0x1)/Pci(0x0,0x0)/Pci(0x0,0x0)/MAC(b48351081138,1)/IPv6([::]:<->[::]:,0,0)..BO
Boot0027  PXE IP4 Intel(R) Ethernet Network Adapter X710-TL PciRoot(0x1)/Pci(0x0,0x0)/Pci(0x0,0x1)/MAC(b48351081139,1)/IPv4(0.0.0.00.0.0.0,0,0)..BO
Boot0028  PXE IP6 Intel(R) Ethernet Network Adapter X710-TL PciRoot(0x1)/Pci(0x0,0x0)/Pci(0x0,0x1)/MAC(b48351081139,1)/IPv6([::]:<->[::]:,0,0)..BO
Boot0029* Diskette Drive    BBS(Floppy,Diskette Drive,0x0)..BO
Boot002A* HGST HUS726T4TALA6L0  BBS(HD,P0: HGST HUS726T4TALA6L0,0x0)..BO
Boot002B* USB Storage Device    BBS(USB,USB Storage Device,0x0)..BO
Boot002C* CD/DVD/CD-RW Drive    BBS(CDROM,CD/DVD/CD-RW Drive,0x0)..BO
Boot002D  Onboard NIC   BBS(Network,IBA CL Slot 00FE v0110,0x0)..BO


============================= Drive/Partition Info =============================

Disks info: ____________________________________________________________________

nvme0n1 : is-GPT,   no-BIOSboot,    has---ESP,  not-usb,    not-mmc, has-os,    no-wind,    2048 sectors * 512 bytes
sda : notGPT,   no-BIOSboot,    has-noESP,  not-usb,    not-mmc, no-os, no-wind,    2048 sectors * 512 bytes

Partitions info (1/3): _________________________________________________________

nvme0n1p3   : is-os,    64, apt-get,    signed grub-efi ,   grub2,  grub-install,   grubenv-ok, update-grub,    farbios
nvme0n1p1   : no-os,    64, nopakmgr,   no-docgrub, nogrub, nogrubinstall,  no-grubenv, noupdategrub,   not-far
sda2    : no-os,    64, nopakmgr,   no-docgrub, nogrub, nogrubinstall,  no-grubenv, noupdategrub,   farbios
sda1    : no-os,    64, nopakmgr,   no-docgrub, nogrub, nogrubinstall,  no-grubenv, noupdategrub,   farbios

Partitions info (2/3): _________________________________________________________

nvme0n1p3   : isnotESP, fstab-has-goodEFI,  no-nt,  no-winload, no-recov-nor-hid,   no-bmgr,    notwinboot
nvme0n1p1   : is---ESP, part-has-no-fstab,  no-nt,  no-winload, no-recov-nor-hid,   no-bmgr,    notwinboot
sda2    : isnotESP, part-has-no-fstab,  no-nt,  no-winload, no-recov-nor-hid,   no-bmgr,    notwinboot
sda1    : isnotESP, part-has-no-fstab,  no-nt,  no-winload, no-recov-nor-hid,   no-bmgr,    notwinboot

Partitions info (3/3): _________________________________________________________

nvme0n1p3   : not--sepboot, with-boot,  fstab-without-boot, not-sep-usr,    with--usr,  fstab-without-usr,  std-grub.d, nvme0n1
nvme0n1p1   : not--sepboot, no---boot,  part-has-no-fstab,  not-sep-usr,    no---usr,   part-has-no-fstab,  no--grub.d, nvme0n1
sda2    : maybesepboot, no---boot,  part-has-no-fstab,  not-sep-usr,    no---usr,   part-has-no-fstab,  no--grub.d, sda
sda1    : maybesepboot, no---boot,  part-has-no-fstab,  not-sep-usr,    no---usr,   part-has-no-fstab,  no--grub.d, sda

fdisk -l (filtered): ___________________________________________________________

Disk nvme0n1: 1.86 TiB, 2048408248320 bytes, 4000797360 sectors
Disk identifier: C926DE13-294F-4E1C-B428-0EFB34E21068
             Start        End    Sectors  Size Type
nvme0n1p1     2048    1742847    1740800  850M EFI System
nvme0n1p2  1742848   18520063   16777216    8G Microsoft reserved
nvme0n1p3 18520064 4000796671 3982276608  1.9T Linux filesystem
Disk sda: 3.64 TiB, 4000787030016 bytes, 7814037168 sectors
Disk identifier: 0x5e6432b3
      Boot      Start        End    Sectors  Size Id Type
sda1             2048 4032045055 4032043008  1.9T 83 Linux
sda2       4032045056 7814035455 3781990400  1.8T 83 Linux

parted -lm (filtered): _________________________________________________________

sda:4001GB:scsi:512:512:msdos:ATA HGST HUS726T4TAL:;
1:1049kB:2064GB:2064GB:ext4::;
2:2064GB:4001GB:1936GB:ext4::;
nvme0n1:2048GB:nvme:512:512:gpt:PC801 NVMe SK hynix 2TB:;
1:1049kB:892MB:891MB:fat32:EFI system partition:boot, esp;
2:892MB:9482MB:8590MB:fat32:Basic data partition:msftres;
3:9482MB:2048GB:2039GB:ext4::;

blkid (filtered): ______________________________________________________________

NAME        FSTYPE   UUID                                 PARTUUID                             LABEL  PARTLABEL
sda                                                                                                   
├─sda1      ext4     d06f911f-9f8f-4218-9e4a-c0327d518b77 5e6432b3-01                                 
└─sda2      ext4     fee50400-d622-4de9-aded-759a562617f5 5e6432b3-02                                 
sdb                                                                                                   
nvme0n1                                                                                               
├─nvme0n1p1 vfat     94F3-F0B1                            cf73badc-18d9-4b5f-afe6-5c279ce0d278 ESP    EFI system partition
├─nvme0n1p2 vfat     8819-913C                            b3c175c3-e3a0-4ef3-bea5-e11c2b133e84 OS     Basic data partition
└─nvme0n1p3 ext4     51139eac-d829-4030-8044-d1328bdfd2d3 fa4918ab-47f2-4c7b-9eb5-de5999d37c73 UBUNTU 

Mount points (filtered): _______________________________________________________

                                               Avail Use% Mounted on
/dev/nvme0n1p3                                  1.6T  10% /
/dev/sda1                                       1.8T   0% /mnt/boot-sav/sda1
/dev/sda2                                       1.6T   0% /mnt/boot-sav/sda2
//storage1.ris.wustl.edu/alex.holehouse/Active  2.8T  89% /work

Mount options (filtered): ______________________________________________________


=================== nvme0n1p1/efi/ubuntu/grub.cfg (filtered) ===================

search.fs_uuid 51139eac-d829-4030-8044-d1328bdfd2d3 root 
set prefix=($root)'/boot/grub'
configfile $prefix/grub.cfg

=================== nvme0n1p2/boot/grub/grub.cfg (filtered) ====================

Install Complete, remove media and reboot.
Dell Recovery

================= nvme0n1p2: Location of files loaded by Grub ==================

           GiB - GB             File                                 Fragment(s)
            ?? = ??             boot/grub/grub.cfg                             1

=================== nvme0n1p3/boot/grub/grub.cfg (filtered) ====================

Ubuntu   51139eac-d829-4030-8044-d1328bdfd2d3
Ubuntu, with Linux 5.19.0-46-generic   51139eac-d829-4030-8044-d1328bdfd2d3
Ubuntu, with Linux 5.15.0-76-generic   51139eac-d829-4030-8044-d1328bdfd2d3
### END /etc/grub.d/30_os-prober ###
UEFI Firmware Settings   uefi-firmware
### END /etc/grub.d/30_uefi-firmware ###
Restore OS to factory state

======================== nvme0n1p3/etc/fstab (filtered) ========================

# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/nvme0n1p3 during installation
UUID=51139eac-d829-4030-8044-d1328bdfd2d3 /               ext4    errors=remount-ro 0       1
# /boot/efi was on /dev/nvme0n1p1 during installation
UUID=94F3-F0B1  /boot/efi       vfat    umask=0077      0       1
/swapfile                                 none            swap    sw              0       0

==================== nvme0n1p3/etc/default/grub (filtered) =====================

GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=menu
GRUB_TIMEOUT=10
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX=""

================= nvme0n1p3: Location of files loaded by Grub ==================

           GiB - GB             File                                 Fragment(s)
1738.973667145 = 1867.208757248 boot/grub/grub.cfg                             1
 158.686183929 = 170.387992576  boot/vmlinuz                                   1
 477.052783966 = 512.231526400  boot/vmlinuz-5.15.0-76-generic                 2
 158.686183929 = 170.387992576  boot/vmlinuz-5.19.0-46-generic                 1
 477.052783966 = 512.231526400  boot/vmlinuz.old                               2
 253.857051849 = 272.576933888  boot/initrd.img                                1
1677.781299591 = 1801.503952896 boot/initrd.img-5.15.0-76-generic              1
 253.857051849 = 272.576933888  boot/initrd.img-5.19.0-46-generic              1
1677.781299591 = 1801.503952896 boot/initrd.img.old                            1

=================== nvme0n1p3: ls -l /etc/grub.d/ (filtered) ===================

-rwxr-xr-x 1 root root 18683 Dec 18  2022 10_linux
-rwxr-xr-x 1 root root 43031 Dec 18  2022 10_linux_zfs
-rwxr-xr-x 1 root root 14387 Dec 18  2022 20_linux_xen
-rwxr-xr-x 1 root root 13369 Dec 18  2022 30_os-prober
-rwxr-xr-x 1 root root  1372 Dec 18  2022 30_uefi-firmware
-rwxr-xr-x 1 root root   700 Feb 20  2022 35_fwupd
-rwxr-xr-x 1 root root   214 Apr 15  2020 40_custom
-rwxr-xr-x 1 root root   215 Dec 18  2022 41_custom
-rwxr-xr-x 1 root root  1362 Jul 13 06:23 99_dell_recovery

==================== nvme0n1p3/etc/grub.d/99_dell_recovery =====================

#!/bin/bash -e
source /usr/lib/grub/grub-mkconfig_lib
cat << EOF
menuentry "Restore OS to factory state" {
        search --no-floppy --hint '(hd0,gpt2)' --set --fs-uuid 8819-913C
        set uuid_options="uuid=8819-913C"
        if [ -s /factory/common.cfg ]; then
            source /factory/common.cfg
        else
            set options="boot=casper automatic-ubiquity noprompt quiet splash nomodeset nopersistent"
        fi
        if [ -s /factory/post-rts-gfx.cfg ]; then
            source /factory/post-rts-gfx.cfg
        fi
        if [ -s /factory/post-rts-wlan.cfg ]; then
            source /factory/post-rts-wlan.cfg
        fi
        #Support starting from a loopback mount (Only support ubuntu.iso for filename)
        if [ -f /ubuntu.iso ]; then
            loopback loop /ubuntu.iso
            set root=(loop)
            set options="iso-scan/filename=/ubuntu.iso \$options"
        fi
        if [ -n "\${lang}" ]; then
            set options="locale=\$lang \$options"
        fi
        if [ -s /factory/dual_enable ]; then
            set options="dell-recovery/dual_boot=true \$options"
        fi
        kernel=/casper/vmlinuz
        if [ ! -f $kernel ]; then
            kernel=/casper/vmlinuz.efi
        fi
        linux   \$kernel dell-recovery/recovery_type=hdd \$uuid_options \$options
        initrd  /casper/initrd
}
EOF



Suggested repair: ______________________________________________________________

The default repair of the Boot-Repair utility would reinstall the grub-efi of
nvme0n1p3,
using the following options:  nvme0n1p1/boot/efi
Additional repair would be performed: unhide-bootmenu-10s use-standard-efi-file

Final advice in case of suggested repair: ______________________________________

Please do not forget to make your UEFI firmware boot on the Ubuntu 22.04.2 LTS entry (nvme0n1p1/efi/****/grub****.efi (**** will be updated in the final message) file) !

You show 4TB drive as DOS? Drives over 2TB must be gpt or they in past used some proprietary work around.

The 4 TB disk is partitioned into two separate partitions each under 2 TBs. No booting should be coming from here (and on a Live USB this disk is automatically mounted without issue when the Live USB starts). Output from gdisk below

sudo gdisk -l /dev/sda

GPT fdisk (gdisk) version 1.0.8

Partition table scan:
  MBR: MBR only
  BSD: not present
  APM: not present
  GPT: not present


***************************************************************
Found invalid GPT and valid MBR; converting MBR to GPT format
in memory. 
***************************************************************

Disk /dev/sda: 7814037168 sectors, 3.6 TiB
Model: HGST HUS726T4TAL
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 6FE8D04B-19DD-4F28-B8E0-6D9A12D769EA
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 7814037134
Partitions will be aligned on 2048-sector boundaries
Total free space is 3693 sectors (1.8 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048      4032045055   1.9 TiB     8300  Linux filesystem
   2      4032045056      7814035455   1.8 TiB     8300  Linux filesystem

Update in response to oldfred (July 24th)

Best to use UUID, not device as that can change.

Agreed, I have changed this back.

But your device entry does not look correct. You do not have p33 or thirty three partitions on NVMe drive.

Correct, sorry this was a typo in my answer which I've fixed

Root partition should also be 1 for pass parameter, so ext4 fsck can be run on boot.

I'm a confused by this comment; where do you see the pass parameter being not set to 1? In /etc/fstab the root partition has the pass parameter set to 1:

UUID=51139eac-d829-4030-8044-d1328bdfd2d3 / ext4 errors=remount-ro 0 1

But maybe I'm missing something elsewhere?

Its drive size over 2TB must be gpt, not each partition. Not sure why you were even able to create that. But many advantages to gpt anyway.

To simply things I've deleted the sda partitions, such that the entire sda device is unallocated. This does not seem to change any of the behavior discussed above, but I figured it might rule out possible things. For completeness, the Boot-Repair output after this is linked here - I realize pasting would be better but the answer body becomes too long if pasted.

What video driver? And is it correctly installed from Ubuntu repository? Old install may have had it, update would not have automatically reinstalled it, and recovery mode uses nomodeset for video.

Using nvidia drivers that in the 5.15.0-76 kernel work absolutely fine (CUDA accessible, nvidia-smi works etc). Drivers are:

NVIDIA-SMI 525.125.06   Driver Version: 525.125.06   CUDA Version: 12.0     |

One thing to note: booting with debug added to the boot line shows that the difference between 5.15.0-76 and 5.19.0-46 is at the point where the boot drive is loaded. Bascically for 5.19 it hangs at

async_tx api initialized (async)

whereas for 5.15 this progress forward to:

EXT4-fs (nvme0n1p3): mounted file system

and then on.

I have messed around a lot with this, including setting rootdelay= to 60 to see if giving it more time will help, but no cigar.

Extra context that brought me here

It took me a lot of hours to get here, so I wanted to share some of the things I did

Initially checked that the 51139eac-d829-4030-8044-d1328bdfd2d3 device was not corrupted. I could boot via a live USB and mount the offending disk just fine. The disk is at /dev/nvme0n1p and the specific Ubuntu OS partition at /dev/nvme0n1p3, as expected for an SSD connected via NVME.

Output of blkid (run after mounting the from the Live USB)

$ blkid

/dev/nvme0n1p1: LABEL="ESP" UUID="94F3-F0B1" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="cf73badc-18d9-4b5f-afe6-5c279ce0d278"
/dev/nvme0n1p2: LABEL="OS" UUID="8819-913C" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="Basic data partition" PARTUUID="b3c175c3-e3a0-4ef3-bea5-e11c2b133e84"
/dev/nvme0n1p3: LABEL="UBUNTU" UUID="51139eac-d829-4030-8044-d1328bdfd2d3" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="fa4918ab-47f2-4c7b-9eb5-de5999d37c73"
/dev/sda1: UUID="d06f911f-9f8f-4218-9e4a-c0327d518b77" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="5e6432b3-01"
/dev/sda2: UUID="fee50400-d622-4de9-aded-759a562617f5" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="5e6432b3-02"
/dev/sdc1: BLOCK_SIZE="2048" UUID="2023-02-23-04-13-44-00" LABEL="Ubuntu 22.04.2 LTS amd64" TYPE="iso9660" PARTLABEL="ISO9660" PARTUUID="a0891d7e-b930-4513-94d8-f629dbd637b2"
/dev/loop1: TYPE="squashfs"
/dev/loop8: TYPE="squashfs"
/dev/loop6: TYPE="squashfs"
/dev/loop4: TYPE="squashfs"
/dev/loop2: TYPE="squashfs"
/dev/loop0: TYPE="squashfs"
/dev/sdc2: SEC_TYPE="msdos" LABEL_FATBOOT="ESP" LABEL="ESP" UUID="F7DB-4D56" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="Appended2" PARTUUID="a0891d7e-b930-4513-94db-f629dbd637b2"
/dev/sdc4: LABEL="writable" UUID="168e1b81-4522-456d-955e-ffae6ecac2c8" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="a0829cd9-4a56-af45-87a1-0dc8d8fdd0ff"
/dev/loop7: TYPE="squashfs"
/dev/loop5: TYPE="squashfs"
/dev/loop3: TYPE="squashfs"

and the output from fdisk is:

sudo fdisk -l

Disk /dev/loop0: 2.54 GiB, 2731876352 bytes, 5335696 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

... skipping loop1-loop7 for brevity

Disk /dev/nvme0n1: 1.86 TiB, 2048408248320 bytes, 4000797360 sectors
Disk model: PC801 NVMe SK hynix 2TB                
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: C926DE13-294F-4E1C-B428-0EFB34E21068

Device            Start        End    Sectors  Size Type
/dev/nvme0n1p1     2048    1742847    1740800  850M EFI System
/dev/nvme0n1p2  1742848   18520063   16777216    8G Microsoft reserved
/dev/nvme0n1p3 18520064 4000796671 3982276608  1.9T Linux filesystem


Disk /dev/sda: 3.64 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: HGST HUS726T4TAL
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x5e6432b3

Device     Boot      Start        End    Sectors  Size Id Type
/dev/sda1             2048 4032045055 4032043008  1.9T 83 Linux
/dev/sda2       4032045056 7814035455 3781990400  1.8T 83 Linux


Disk /dev/sdc: 7.5 GiB, 8053063680 bytes, 15728640 sectors
Disk model: Flash Disk      
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: A0891D7E-B930-4513-94D9-F629DBD637B2

Device       Start      End Sectors  Size Type
/dev/sdc1       64  9613459 9613396  4.6G Microsoft basic data
/dev/sdc2  9613460  9623527   10068  4.9M EFI System
/dev/sdc3  9623528  9624127     600  300K Microsoft basic data
/dev/sdc4  9625600 15728576 6102977  2.9G Linux filesystem


Disk /dev/loop8: 346.33 MiB, 363151360 bytes, 709280 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

And the content of fstab on the OS disk is

# note mount point defined from the live USB
cat /mnt/mydisk/etc/fstab

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/nvme0n1p3 during installation
UUID=51139eac-d829-4030-8044-d1328bdfd2d3 /               ext4    errors=remount-ro 0       1

# /boot/efi was on /dev/nvme0n1p1 during installation
UUID=94F3-F0B1  /boot/efi       vfat    umask=0077      0       1
/swapfile                                 none            swap    sw              0       0

#UUID=d06f911f-9f8f-4218-9e4a-c0327d518b77 /mnt/local1  ext4 defaults  0  0
#UUID=fee50400-d622-4de9-aded-759a562617f5 /mnt/local2  ext4 defaults  0  0

My boot mode is set to UEFI: Secure Boot Off, and from the BIOS I had checked the enable Legacy Boot ROM to load the USB mounter.

I also tried updating fstab to mount via the actual path (i.e. changing

UUID=51139eac-d829-4030-8044-d1328bdfd2d3 / ext4 errors=remount-ro 0 1

To

/dev/nvme0n1p3 / ext4 errors=remount-ro 0 1 (NB this previously was written as nvme0n1p33 which was a typo on the answer)

Which also does not make a difference - loading in 5.15.0.76 works either way, and loading with 5.19.0.76 fails either way.

oldfred avatar
cn flag
Please copy & paste the pastebin link to the BootInfo summary report ( do not post report), do not run the auto fix till reviewed. Use often updated ppa version over somewhat older ISO with your USB installer or any working install. https://help.ubuntu.com/community/Boot-Repair & https://sourceforge.net/p/boot-repair/home/Home/ You may need advanced mode. But not sure if related or not. You show 4TB drive as DOS? Drives over 2TB must be gpt or they in past used some proprietary work around. What does `sudo gdisk -l /dev/sda` show?
ash avatar
lk flag
ash
Updated answer in response (see the 'updated' for today's dat) - thanks so much!
oldfred avatar
cn flag
Best to use UUID, not device as that can change. But your device entry does not look correct. You do not have p33 or thirty three partitions on NVMe drive. Root partition should also be 1 for pass parameter, so ext4 fsck can be run on boot. Its drive size over 2TB must be gpt, not each partition. Not sure why you were even able to create that. But many advantages to gpt anyway. What video driver? And is it correctly installed from Ubuntu repository? Old install may have had it, update would not have automatically reinstalled it, and recovery mode uses nomodeset for video.
ash avatar
lk flag
ash
Updated in response to questions and also identified final step in boot the shows in the debug logs
oldfred avatar
cn flag
I am not seeing anything in Boot-Repair report, others may find something. But back to nVidia driver. Is nVidia driver installed with newer kernel. Generally have to reinstall driver with upgrade so newer kernel has it. #What is installed `dkms status`
ash avatar
lk flag
ash
Hmm - so not familiar with `dkms`, but running `dkms status` does not return anything (no error; `echo $?` after returns `0`). Also, thank you so so much for your help here; greatly appreciated and has been really helpful for me to rule things in/out!
oldfred avatar
cn flag
I think that means you do not have nVidia driver installed or installed correctly from Ubuntu repository. [purge & install nvidia](https://askubuntu.com/questions/1418211/what-happened-to-my-installed-driver) & https://askubuntu.com/questions/813676/installing-ubuntu-mate-with-dual-boot-option-on-windows-10-usb-booting-not-hap/814413#814413
ash avatar
lk flag
ash
So I want to thank you for all your help here. Whether or not the graphics card issue is the source of my original problem I don't know; the drivers are installed and I can run/compile cuda, but I did also realize that gnome was not being output via the GPU and was falling back to the CPU. This was the last straw for me. I have given up and reset back to 20.04, which works fine. FWIW, in 20.04 `dkms` also returns nothing, but now gnome-shell is running on the GPU as per `nvidia-smi`, and I am content things are in 20.04 working as they should.
ash avatar
lk flag
ash
A frustrating end to 20+ hours of time, but I learned a lot in the process (chiefly among them: contact Dell before you dist-upgrade a Dell-purchased Ubuntu machine. As a final note for anyone following this: post re-install of 20.04 via the Dell recovery package, I hit a Secure Boot blue screen of death, presumably because the key on the original factory version is now out of date vs. an updated secure boot. I got around this by disabling Secure Boot in the BIOS (F12 on boot). May re-enable but having booted into BIOS 100+ times in the last weeks may give it a minute...
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.