Score:0

Help me choose a drive pooling strategy for my NAS / media server

my flag

I'm currently running a NAS with a Ryzen 1600, 16GB of memory, Ubuntu Server, a small SSD for the OS and two 8TB HDDs (mounted at /data and /backup). Both drives are ext4 formatted and about 60% filled up.

/data is holding all my data including nextcloud userdata and movies/tv-series many of which are hardlinked to different locations on the drive. I use Rsnapshot to do nightly backups from /data to /backup.

I like this setup mainly because it's pretty simple and straight forward but I think it lacks an upgrade path that keeps the simplicity. For example if I wanted to add more drives in the future (e.g. two 14TB drives) I could just mount them at /data2 and /backup2 but I think that's a very inconvenient solution.

What are my options here?

I was reading about btrfs across multiple devices and I kind of like the idea to create a btrfs pool with the /data drive and a new 14TB drive, likewise I would expand /backup by another 14TB and create a second btrfs pool. That way I can keep my Rsnapshot backup solution working and add even more drives in the future. The downside to this, as I understand, would be that hardlinks won't work anymore because data will be striped across the 8TB and 14TB drives. Is that assumption correct?

I was also looking into RAID 5/6 but to my knowledge that would restrict future drive purchases to the 8TB already present. Also I feel more comfortable having two separate drives/pools for /data and /backup instead of dealing with parity drives etc.

Please tell me if that idea in general is feasible or if there are better ways to approach this.

Score:0
in flag

To answer this question in a manner that reduces the risk of personal opinions, I'll try to keep this dry:

Q. What are my options here?

Ubuntu comes with a number of solutions out of the box:

  1. Logical Volumes

    These are a reliable storage management technology that gives people the ability to group storage devices – or different partitions within the same storage device – into what appears to be a single volume/partition. Volumes can be grown and shrunk on an as-needed basis.

    Ubuntu supports this out of the box.

  2. ZFS Storage Pools

    To quote Wikipedia, ZFS combines a file system with a volume manager. This allows you to grow and shrink the size of the storage pool on an as-needed basis and a lot of additional benefits, such as file system replication, protection against data corruption, and more.

    Ubuntu 21.10 supports this quite well, and it's still considered "Experimental" with 20.04 LTS.

  3. BTRFS

    This has a lot of similarities with ZFS, though goes about them in different ways. You can add and remove storage devices to the pool with a few commands and the community surrounding the file system is quite passionate.

    Some packages will need to be installed to use BTRFS on Ubuntu, and the various tutorials online will make this a quick little project.

  4. RAID

    As you've already surmised, RAID will allow you to put a bunch of disks together to appear as a larger one, with JBOD (Just a Bunch Of Disks) being one common method for people who add and remove storage in an ad hoc fashion. Unlike ZFS and BTRFS, though, there can be no expectation for protection from data loss in the event of a disk failure or bit rot. If you value your data and really want to use RAID for a large storage pool, a complete array must be planned before its construction and you'll want to go with RAID5 or RAID6 to maximise storage while limiting risk of data loss.

    As one would expect, Ubuntu supports this out of the box.

There are likely others, but these four are the most common that are discussed when building home storage servers. Of the four, Logical Volumes are arguably the simplest to implement. ZFS and BTRFS are the better choices if you want to keep your data for decades while slowly migrating to newer disks over time. RAID is an option.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.