Is it safe to use dmcrypt on top of software raid?

kd8azz

2/18/23, 8:00 PM

I'm designing a machine, to serve as a NAS. My interest is hobby-level.

I would use mdadm to software raid a couple of same-size hdds. At first I'd use raid1, but in the future I'd add another disk and convert it to raid5. My understanding is that mdadm can do that in-place. (Yes, I know I should back the data up anyway, first.)

I would use bcache with this raid array, and an ssd, to improve performance. (bcache is a block device caching layer)

I'd put dmcrypt on top of it.

|       dmcrypt     |              |          dmcrypt         |
|       bcache      |      ->      |          bcache          |
      | mdadm raid1 |                    |    mdadm raid5     |
| ssd | hdd1 | hdd2 |              | ssd | hdd1 | hdd2 | hdd3 |

I think the only effect this arrangement would have on dmcrypt is to store the dmcrypt header with parity, and cache it to the ssd if it's used often enough. Am I correct that this is safe? Or is there some unknown-to-me issue here, that would cause data loss?

0 + 0

mdadm

dmcrypt

Mike Andrews

2/19/23, 1:24 PM

A three disk RAID5 has no capacity advantage over a three disk RAID1E. Both can store 2x the base disk size. You'll be far happier with the performance of the 1E and it has a far lower risk of ruining your day at some point in the future. The "funny" part about bcache is that if you don't use bcache in `writethrough` mode, that SSD has to be there, because it's caching the writes, so you don't really have data redundancy. If you use it in `writethrough`, you've lost any write acceleration benefits.

kd8azz

2/19/23, 2:59 PM

Thanks! I didn't know about RAID1E. Re: bcache, I guess I assumed that writeback mode would give me data redundancy at any point when the NAS had been idle for more than a couple minutes. Maybe I've got my brain stuck in a file-oriented world. Would the loss of a given block ruin the whole volume?

Mike Andrews

2/20/23, 8:26 PM

In my experimentation with bcache, I couldn't get it to reliably flush every last bit of data during idle periods. However, maybe it's better now! With bcache under the dmcrypt layer, a lost block very well could ruin the whole volume. Before you take on bcache, try to come up with some cases where you think it'd help your use case. My home NAS, for instance, is there for serving up media files. I can't say that it would really benefit from bcache: I'm unlikely to play the same media twice in a row. And, price out flash drives too... use QLC if your workload is mostly reads.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Is it safe to use dmcrypt on top of software raid?

TH: ปลอดภัยไหมที่จะใช้ dmcrypt เหนือการจู่โจมซอฟต์แวร์

RO: Este sigur să utilizați dmcrypt pe lângă raidul software?

RU: Безопасно ли использовать dmcrypt поверх программного рейда?

VI: Có an toàn khi sử dụng dmcrypt trên phần mềm đột kích không?

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.