Score:1

How to add a new drive to a RAID (mdadm), when that RAID is encrypted (LUKS), and makes up a physical volume in a volume group (LVM)?

us flag

I'm looking to add a new drive to an existing NAS server, and would appreciate a second look at my plan before I pull the trigger. Part of the challenge is the new disk will be added to an existing RAID, which itself is encrypted, and also makes up a physical-volume in an existing volume-group.

Current Setup:

There is a single logical-volume ('media') and volume-group ('raid') made up of two physical-volumes ('/dev/mapper/crypt1' and '/dev/mapper/crypt2'). Each physical-volume is a LUKS-encrypted RAID: a (1) RAID-5 made up of 4x4TB drives ('md0'), and a (2) RAID-1 made up of 2x8TB drives ('md127').

user@server:~$ lsblk

sdb                8:16   0   3.7T  0 disk  
└─md0              9:0    0  10.9T  0 raid5 
 └─crypt1        253:0    0  10.9T  0 crypt 
   └─raid-media  253:2    0  18.1T  0 lvm   /media
sdc                8:32   0   3.7T  0 disk  
└─md0              9:0    0  10.9T  0 raid5 
 └─crypt1        253:0    0  10.9T  0 crypt 
   └─raid-media  253:2    0  18.1T  0 lvm   /media
sdd                8:48   0   3.7T  0 disk  
└─md0              9:0    0  10.9T  0 raid5 
 └─crypt1        253:0    0  10.9T  0 crypt 
   └─raid-media  253:2    0  18.1T  0 lvm   /media
sdf                8:80   0   3.7T  0 disk  
└─md0              9:0    0  10.9T  0 raid5 
 └─crypt1        253:0    0  10.9T  0 crypt 
   └─raid-media  253:2    0  18.1T  0 lvm   /media
sdg                8:96   0   7.3T  0 disk  
└─md127            9:127  0   7.3T  0 raid1 
 └─crypt2        253:1    0   7.3T  0 crypt 
   └─raid-media  253:2    0  18.1T  0 lvm   /media
sdh                8:112  0   7.3T  0 disk  
└─md127            9:127  0   7.3T  0 raid1 
 └─crypt2        253:1    0   7.3T  0 crypt 
   └─raid-media  253:2    0  18.1T  0 lvm   /media

Objective:

I would like to add a third 8TB disk to the 'md127' array without losing the existing data. And since this will grow the array from two drives to three, I would like to reshape the array from RAID-1 to RAID-5 for a net gain of approximately 8TB of free storage.

Step 0: Unmount the Logical Volume, and Convert RAID-1 to RAID-5

unmount /dev/raid/media
mdadm --grow /dev/md127 --level=5

Step 1: Add HDD to RAID-5

Supposing the new drive is added as '/dev/sda':

$ mdadm --add /dev/md127 /dev/sda
$ mdadm --grow --raid-devices=3 --backup-file=~/md127_grow.bak /dev/md127

I understand this can take a long time. One can check the progress with mdadm --detail /dev/md0.

Step 2: Enlarge the physical-drive, volume-group, and filesystem

pvresize /dev/mapper/crypt2 
lvextend -l +100%FREE /dev/raid/media
resize2fs /dev/raid/media

I took my notes from this page and this page, neither of which consider any encryption. My plan above doesn't consider encryption, except when identifying the physical volume (e.g. '/dev/mapper/crypt2'). I'm guessing that's the only thing I do differently.

I would appreciate any thoughts or comments you may have. Many thanks for the help.

in flag
There is a lot that can go wrong here if the CPU on the NAS is overtaxed or if the system runs out of memory. While the steps you’ve compiled should work *in theory*, I would strongly advise having a backup of the most important data first, even if it is just a temporary upload to a private S3 bucket. I’ve ruined many weekends in my youth while modifying RAID sets in an attempt to “not lose anything” only to lose it all anyway
Aboottogo avatar
us flag
@matigo A backup would be ideal, but my 4Mbps upload speed would take way too long to upload 18TB to the cloud. I looked into it, and was surprised I couldn't just rent a a second NAS for the week to make a temporary on-site backup. Hence why I'm really nervous to pull the trigger.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.