Join Log in
Score:0
avatar Server

Why does this EFI-based Linux host keep booting to the second drive?

ls flag
rlpowell

I have several servers with the same basic setup: primary drive with VFAT /boot/efi, XFS /boot, and XFS / on LVM; and secondary drive with exactly the same thing, each partition rsynced nightly.

I also have a script that makes sure (theoretically) that grub and EFI and so on are set up so that when the server is rebooted, it boots from the primary drive. Normally this works, but on this particular server, it seems to always prefer the secondary drive, even when I swap drives. That is: it seems to consistently boot the drive in the second slot (although it's worth noting that since I cabled the drives myself, it's entirely possible that the second slot actually enumerates first on the motherboard, but I can't see why that would even matter).

Here's the LVM situation:

rlpowell@stodi> sudo pvs
  PV         VG                   Fmt  Attr PSize   PFree
  /dev/sda3  stodi_orange_2021_10 lvm2 a--  930.55g    0
  /dev/sdb3  stodi_pink_2020_10   lvm2 a--  930.55g    0
rlpowell@stodi> sudo lvs
  LV   VG                   Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root stodi_orange_2021_10 -wi-ao---- 930.55g
  root stodi_pink_2020_10   -wi-ao---- 930.55g

(On this host, orange is currently primary and pink is currently secondary.)

When it's up and running we have:

/dev/mapper/stodi_orange_2021_10-root  931G  686G  245G  74% /
/dev/sda2                              483M  256M  228M  53% /boot
/dev/sda1                              488M  6.4M  482M   2% /boot/efi

rlpowell@stodi> cat /proc/partitions
major minor  #blocks  name

   8        0  976762584 sda
   8        1     499712 sda1
   8        2     499712 sda2
   8        3  975762119 sda3
  11        0    1048575 sr0
 253        0  975757312 dm-0
   8       16  976762584 sdb
   8       17     499712 sdb1
   8       18     499712 sdb2
   8       19  975762119 sdb3
 253        1  975757312 dm-1

Here's the EFI situation:

rlpowell@stodi> sudo efibootmgr -v
BootCurrent: 0013
Timeout: 30 seconds
BootOrder: 0001,0015,000C,0000,0013,0002,0003,0004,0005,0006,0007,0008,0009,000A,000B,000C,000D,000E,000F,0010,0011,0012,0013,0014,0016
Boot0000  Startup Menu  FvVol(cdbb7b35-6833-4ed6-9ab2-57d2acddf6f0)/FvFile(9d8243e8-8381-453d-aceb-c350ee7757ca)....ISPH
Boot00000013* Samsung SSD 870 EVO 1TB   PciRoot(0x0)/Pci(0x11,0x4)/Sata(0,0,0)N.....YM....R,Y.....ISPH
Boot0001* stodi_orange_2021_10  HD(1,GPT,cb8227c2-dc52-fc4f-bd8f-b46f71104428,0x800,0xf4000)/File(\EFI\fedora\shimx64.efi)
Boot0002  Bios Setup    FvVol(cdbb7b35-6833-4ed6-9ab2-57d2acddf6f0)/FvFile(9d8243e8-8381-453d-aceb-c350ee7757ca)......ISPH
Boot0003  3rd Party Option ROM Management       FvVol(cdbb7b35-6833-4ed6-9ab2-57d2acddf6f0)/FvFile(9d8243e8-8381-453d-aceb-c350ee7757ca)......ISPH
Boot0004  System Diagnostics    FvVol(cdbb7b35-6833-4ed6-9ab2-57d2acddf6f0)/FvFile(9d8243e8-8381-453d-aceb-c350ee7757ca)......ISPH
Boot0005  System Diagnostics    FvVol(cdbb7b35-6833-4ed6-9ab2-57d2acddf6f0)/FvFile(9d8243e8-8381-453d-aceb-c350ee7757ca)......ISPH
Boot0006  System Diagnostics    FvVol(cdbb7b35-6833-4ed6-9ab2-57d2acddf6f0)/FvFile(9d8243e8-8381-453d-aceb-c350ee7757ca)......ISPH
Boot0007  System Diagnostics    FvVol(cdbb7b35-6833-4ed6-9ab2-57d2acddf6f0)/FvFile(9d8243e8-8381-453d-aceb-c350ee7757ca)......ISPH
Boot0008  Boot Menu     FvVol(cdbb7b35-6833-4ed6-9ab2-57d2acddf6f0)/FvFile(9d8243e8-8381-453d-aceb-c350ee7757ca)......ISPH
Boot0009  USB:          BBS(65535,,0x0)/PciRoot(0x0)/Pci(0x1d,0x0)......ISPH
Boot000A  Network Boot  FvVol(cdbb7b35-6833-4ed6-9ab2-57d2acddf6f0)/FvFile(9d8243e8-8381-453d-aceb-c350ee7757ca)......ISPH
Boot000B* Samsung SSD 870 EVO 1TB       BBS(HD,Harddisk1,0x400)/PciRoot(0x0)/Pci(0x11,0x4)......ISPH
Boot000C* IBA GE Slot 00C8 v1550        BBS(Network,Network1,0x0)/PciRoot(0x0)/Pci(0x19,0x0)......ISPH
Boot000D* IBA GE Slot 0500 v1550        BBS(Network,Network1,0x0)/PciRoot(0x0)/Pci(0x1c,0x0)/Pci(0x0,0x0)......ISPH
Boot000E* IBA GE Slot 0500 v1550        BBS(Network,Network1,0x0)/PciRoot(0x0)/Pci(0x1c,0x0)/Pci(0x0,0x0)......ISPH
Boot000F  USB:          PciRoot(0x0)/Pci(0x1d,0x0)N.....YM....R,Y.....ISPH
Boot0010* hp DVDRW DU8A6SH      PciRoot(0x0)/Pci(0x1f,0x2)/Sata(0,0,0)N.....YM....R,Y.....ISPH
Boot0011* hp DVDRW DU8A6SH      BBS(CDROM,CDROM1,0x400)/PciRoot(0x0)/Pci(0x1f,0x2)......ISPH
Boot0012  System Information    FvVol(cdbb7b35-6833-4ed6-9ab2-57d2acddf6f0)/FvFile(9d8243e8-8381-453d-aceb-c350ee7757ca)......ISPH
Boot0013* Fedora        HD(1,GPT,cb8227c2-dc52-fc4f-bd8f-b46f71104428,0x800,0xf4000)/File(\EFI\fedora\shimx64.efi)....ISPH
Boot0014  HP Recovery   FvVol(cdbb7b35-6833-4ed6-9ab2-57d2acddf6f0)/FvFile(9d8243e8-8381-453d-aceb-c350ee7757ca)......ISPH
Boot0015* stodi_pink_2020_10    HD(1,GPT,3dc251c1-c845-1f4b-9a0b-ebc15792efd6,0x800,0xf4000)/File(\EFI\fedora\shimx64.efi)
Boot0016* Samsung SSD 860 EVO 1TB       PciRoot(0x0)/Pci(0x11,0x4)/Sata(1,0,0)N.....YM....R,Y.....ISPH

And here's grub:

rlpowell@stodi> sudo grub2-editenv /boot/grub2/grubenv list
saved_entry=a7afe39309446d92f17c60bc5467f619-5.14.9-200.fc34.x86_64
kernelopts=root=/dev/stodi_orange_2021_10/root ro rd.auto enforcing=0 crashkernel=auto video=DP-1:1280x1024-32@60e
boot_success=1
boot_indeterminate=0

rlpowell@stodi> sudo cat /boot/loader/entries/a7afe39309446d92f17c60bc5467f619-5.14.9-200.fc34.x86_64.conf
title Fedora (5.14.9-200.fc34.x86_64) 34 (Thirty Four)
version 5.14.9-200.fc34.x86_64
linux /vmlinuz-5.14.9-200.fc34.x86_64
initrd /initramfs-5.14.9-200.fc34.x86_64.img
options root=/dev/mapper/stodi_orange_2021_10-root ro rd.auto enforcing=0 crashkernel=auto video=DP-1:1280x1024-32@60e
grub_users $grub_users
grub_arg --unrestricted
grub_class kernel

But if I reboot, as I said, it'll reliably end up running on the pink LV instead (i.e. on the secondary drive). This is true even when I previously booted from orange with the pink drive removed.

I can't see what part of this makes that possible.

Maybe there's something going on in the BIOS that efibootmgr can't affect?

EDIT: UUID info:

rlpowell@stodi> sudo blkid
/dev/sda1: SEC_TYPE="msdos" UUID="E6B6-7A41" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="/boot/efi - EFI System Partition" PARTUUID="cb8227c2-dc52-fc4f-bd8f-b46f71104428"
/dev/sda2: UUID="544b6c29-f436-4b2f-ab73-f2cf643638b7" BLOCK_SIZE="512" TYPE="xfs" PARTLABEL="/boot Partition" PARTUUID="fe677923-076f-2c47-bf0d-98b4d7e3e86f"
/dev/sda3: UUID="ur5eed-EJqk-y2op-F1dY-RpUC-SQGW-MCUk48" TYPE="LVM2_member" PARTLABEL="LVM: root on stodi_orange_2021_10" PARTUUID="b4754fd9-f637-2b4c-9fec-3e120521956f"
/dev/mapper/stodi_orange_2021_10-root: UUID="011f7198-26d2-4c34-bc39-85fc45c55422" BLOCK_SIZE="512" TYPE="xfs"
/dev/sdb1: SEC_TYPE="msdos" UUID="6CBC-8CFB" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="/boot/efi - EFI System Partition" PARTUUID="3dc251c1-c845-1f4b-9a0b-ebc15792efd6"
/dev/sdb2: UUID="0719024b-8442-4b64-a279-842a6533b511" BLOCK_SIZE="512" TYPE="xfs" PARTLABEL="/boot Partition" PARTUUID="8b0e313c-37bc-d242-8b23-b4fc6dbaf519"
/dev/sdb3: UUID="82fp7k-I0Th-92EP-gJ0O-MD5A-efK0-YxZ0bP" TYPE="LVM2_member" PARTLABEL="/ Partition" PARTUUID="b64d042d-443e-6445-afbf-0673810b5a2a"
/dev/mapper/stodi_pink_2020_10-root: UUID="ac593057-5586-416c-b141-c8173cdb4d61" BLOCK_SIZE="512" TYPE="xfs"
85
0 + 0
in flag
NiKiZe
this goes mostly on guids, EFI boot mgr identifies it's entries on GPT guids if possible. Maybe add blkid / lsblk and sfdisk -d for both disks as well. check if any guid is same between drives. Maybe also compare this to your other machines where it behaves differently. To narrow this down further, try to figure out exactly which drive is used in what stage of the boot process, in theory, firmware could load grub from one drive, grub config loaded from the other drive etc.
1
Reply
ls flag
rlpowell
Added blkid info; it appears to match correctly. Do you have suggestions on how to figure out which drive is used in which stage?
0
Reply
in flag
NiKiZe
Can you boot the machine with only one drive, and does it work with both drives (one by one)?
0
Reply
ls flag
rlpowell
Yes, it will boot from either drive if they're the only drive in the box (that's how I'm handling the problem, in fact; boot with the secondary drive popped out, pop it in after early boot).
0
Reply
Ondrej Simek avatar
mx flag
Ondrej Simek
Is it a Supermicro by any chance? Because those have [some issues](https://github.com/rhboot/efibootmgr/issues/19#issuecomment-595516274). I wasn't even able to change the order through BIOS and had to just remove the OS I didn't want to boot into.
0
Reply
Score:0
avatar Server
ls flag
rlpowell

What seems to have worked is deleting every entry with "GPT" in it before I add my own:

    for num in $(efibootmgr -v | grep GPT | sed -r 's/^Boot0*([0-9A-F]+).*/\1/')
    do
        efibootmgr -b $num -B
    done
+ 0
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.