Score:2

Storage Spaces Direct CSV vs non-CSV volume performance difference

br flag

In a Win2019 S2D deployment having 4 nodes of 4 SSD per node, I create a new NTFS volume and I create a new CSV_NTFS volume. I'm using diskspd to test. I'm running the tests from the node in which I will obtain DirectIO.

I'm getting 250k/610k write/read 4k IOPs on the NTFS and 187k/954k IOPs on the CSV. I would think the underlying subsystem is essentially identical, except that the NTFS volume is shared in the case of CSV.

Why would read performance be markedly better when the volume is using CSVFS or is this an anomaly?

Score:4
kz flag

Bringing in distributed lock / arbitration layer (CSVFS) on top of the NTFS/ReFS comes with a performance penalty.

TL;DR: Everything works as expected.

W Lucking avatar
br flag
I guess my post is confusing. I'm asking why there is a read performance gain while using CSVFS. How would CSVFS cause a gain in read performance over its non-use?
BaronSamedi1958 avatar
kz flag
CSV Cache and partner nodes participating in read I/O (data is getting delivered from the multiple nodes @ the same time). Turn CSV Cache OFF and put down all the partner nodes to re-run the test and see would it make any difference.
W Lucking avatar
br flag
Ok, you are right. While diskspd is often run with "cache disabled", the term "cache" here does not include CSV cache (I assumed it had). I needed to turned off CSV cache (BlockCacheSize=0) to perform a meaningful comparison. Thanks.
BaronSamedi1958 avatar
kz flag
It’s not only that. SMB Multichannel and reading replicas from the other nodes should be disabled as well.
W Lucking avatar
br flag
Why should SMB Multichannel be disabled too? According to the documentation you aren't going to get the use of RDMA (which I use in my network) unless you have this enabled.
BaronSamedi1958 avatar
kz flag
Because you’ll read the data from the partner nodes rendering whole set of your test runs basically useless.
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.