Score:2

Are affordable hardware raids with BBU still worth it?

bo flag

We have done a lot of research and got conflicting answers from experts. What is your opinion on the following?

  • 14 x 16 TB HDDs (Raid 10 -> 100 TB usable)
  • HW: SGL 8405 + AEC-82885T + BBU (replacement kept in stock)
  • 16 thread CPU & 256 GB ECC Memory

We need to build our own replicated S3 (minio) setup for big data cold storage. We spoke with experts on the matter of hardware vs software raid and got very conflicting answers. None could give strong explanations.

What is the state nowadays with HW vs madam raids? Is it still worth it to invest in HW for the main concern of reliability?

Any tips on how to pick which way to go?

paladin avatar
id flag
When using a hardware raid, the system hardware might be more expensive, but managing such system as an admin is more easier and thus you are saving money in workforce. For software raid you also need a performant CPU and additional system memory for good performance. Another software raid technique is provided by btrfs filesystem. I would give it a try, it's a bit different to mdadm but has many advantages. PS do not use raid5/6 modes of btrfs, as those are experimental and unstable. But raid1 and raid10 are stable.
paladin avatar
id flag
PS When using big raids you should usually go for scsi and when doing so, good scsi controllers are usually raid controllers. So going for hardware raid is usually the way to go. On the other hand, software raid systems allow more customization.
user2693017 avatar
bo flag
@paladin can you explain why?
user2693017 avatar
bo flag
@paladin what do you mean with maintenance time? madam is super simple and easy to maintain in my experience. We use it everywhere where we have SSDs or NVMEs.
paladin avatar
id flag
An hardware raid is OS independent for example. OS independence also means that such system is more bullet proof vs cyber criminal attacks (at least when you can trust the manufacturer). Setting up a software raid takes more time and more maintenance than setting up a hardware raid (I'm also talking about installing the OS, configuring it proper and all the etc.). Hardware raid is usually faster when rebuilding the raid, hardware raids are also usually faster than software raids, especially with many drives. - This all saves time and money.
user2693017 avatar
bo flag
I see. It is not used as boot drive and if it would a preconfigured install image would take care of it all ;) Thank you for explaining.
paladin avatar
id flag
PPS when using software raid, don't use mdraid, as it's outdated for big arrays. Better use a filesystem with implemented raid function, like ZFS or BTRFS. As those extraordinary reduce rebuild time and are easier to backup. (Especially BTRFS is very easy to backup.)
user2693017 avatar
bo flag
Thank you, @paladin
Score:0
ca flag

Short answer: I would use ZFS (with striped mirrors) and call it a day.

Alternative answer: if you really want a traditional hardware RAID setup, go with some modern LSI-based cards as the ones based on Broadcom/LSI SAS3516 or SAS3916.

Be aware that AEC-82885T seems a plain SAS expander card, with no RAID or BBU (as a side note, it should be a good card for a ZFS SAS setup).

user2693017 avatar
bo flag
Thank you. The data that is stored is already compressed and de-duplicated. Therefore ext4 seems to be the better pick. Unfortunately only AEC-82885T is possible and yes the BBU is additional (not part of the card).
user2693017 avatar
bo flag
I forgot, SGL 8405 is the other part.
shodanshok avatar
ca flag
Adaptec SGL 8405 is a completely different card, a real HW RAID controller. Do you want to connect the SGL card to the 82885T one? Anyway, with such amount of data, I would reiterate that ZFS is the correct choice (compression and dedup apart).
user2693017 avatar
bo flag
I don't know how they are being connected but that is what the data center is offering/using as the only option (these two combined). So I suppose ZFS would have faster rebuild times than the HW with those disks?
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.