LVM mirror with failed disk can't be repaired

tj flag

I'm trying to help someone resurrect a RAID1 mirrored LVM logical volume on an old server running Fedora 20 (lvm2-, linux 3.19.8-100). One of the disks in the mirror is dead. The disk belonged to a VG named systemvg. vgchange says:

vgchange -a y --partial systemvg
  PARTIAL MODE. Incomplete logical volumes will be processed.
  PV qtva6m-iBhc-q3Zt-At1U-FiWH-iZd8-tjpR0u not recognised. Is the device missing?
  device-mapper: reload ioctl on  failed: Input/output error
  1 logical volume(s) in volume group "systemvg" now active

(The "ioctl on failed" is puzzling, but whatever.)

Now we try to repair the LV named "home" in that VG:

lvconvert --repair systemvg/home
  PV qtva6m-iBhc-q3Zt-At1U-FiWH-iZd8-tjpR0u not recognised. Is the device missing?
Attempt to replace failed RAID images (requires full device resync)? [y/n]: y
  device-mapper: create ioctl on systemvg-home_rmeta_2 failed: Device or resource busy
  Failed to suspend systemvg/home before committing changes
  Failed to replace faulty devices in systemvg/home.
  Device '/dev/sdc' has been left open (1 remaining references).
  Device '/dev/sda1' has been left open (1 remaining references).
  Internal error: 2 device(s) were left open and have been closed.

The situation is now complicated by the fact that we (foolishly?) added a new PV (new blank disk) to systemvg using vgextend, in the hope that this would help. It didn't, and now we can't remove that disk from systemvg.

lsblk currently shows:

sda                                  8:0    0  7.3T  0 disk 
└─sda1                               8:1    0  7.3T  0 part 
  ├─systemvg-home_rmeta_2          253:9    0    4M  0 lvm  
  └─systemvg-home_rimage_2         253:10   0  5.5T  0 lvm  
sdc                                  8:32   0  5.5T  0 disk 
├─systemvg-home_rmeta_0            253:2    0    4M  0 lvm  
└─systemvg-home_rimage_0           253:3    0  5.5T  0 lvm  
systemvg-home_rmeta_1-missing_0_0  253:4    0    4M  0 lvm  
└─systemvg-home_rmeta_1            253:5    0    4M  0 lvm  
systemvg-home_rimage_1-missing_0_0 253:6    0  5.5T  0 lvm  
└─systemvg-home_rimage_1           253:7    0  5.5T  0 lvm  

sdc is the remaining original half of the mirror. sda is the new empty disk we added to systemvg.

/dev/mapper contains:


but no "/dev/mapper/systemvg-home".

How can we recover the data from the remaining half of the mirror?

I sit in a Tesla and translated this thread with Ai:


Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.