Storing each file on separate physical drives for a single file volume

KT8

7/21/24, 7:39 AM

I am working with a bunch of really large files (>100 GB). As we often process multiple large files in parallel (e.g. 10-12), we think that it might be advantageous to have each file be stored on separate physical spinning drives, rather than have the data stripped across multiple physical drives (e.g. with raid) where the software would then be forced to perform a bunch of random seek operations when processing all these files in parallel.

The Oracle Sun QFS file system seems to allow one to join multiple harddrives together as a single volume, while storing each file on separate physical drives with a round-robin allocation method (https://docs.oracle.com/cd/E22586_01/html/E22571/gkwrn.html). This would therefore fulfill our needs. I was wondering if there might be an equivalent file system for Ubuntu/Debian etc. that can store each file on separate physical drives while allowing these drives to be used as a single volume?

1 + 1

raid

ubuntu

storage

filesystems

lvm

Nikita Kipriyanov

7/21/24, 9:03 AM

Product recommendations are off topic on ServerFault. However, did you look *the reverse*, e.g. virtual file systems that able to fuse several physical file systems into a single virtual space? Like https://github.com/trapexit/mergerfs which allows for different allocation policies, e.g. where to physically put files. Be aware that this layer itself will somewhat degrade performance.

Score:2

Server

shodanshok

7/21/24, 12:55 PM

If files are on different directory, you can try creating concatenated volume with a plain XFS filesystem on top, tuning XFS allocation group behavior to match what you want.

That said, I would simply stripe drives using a big enough (512 KB) chunk size: in this manner you can use the familiar RAID approach while minimizing seek penalty.

Anyway, be sure to test the chosen setup with some real-world workloads, as storage performance can be surprising at times.

+ 0

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Storing each file on separate physical drives for a single file volume

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.