Score:0

Should Hard drives always set the rotatinonal flag in /sys/block//queue/rotational?

cn flag

Folks,

AWS exposes HDDs in their D3 instances (https://aws.amazon.com/ec2/instance-types/d3/) as nvme drives and these drives DON'T set the rotational flag as they're supposed to (https://lwn.net/Articles/408428/).

I have cut a ticket to AWS, but their response was that this was automatically set by the linux kernel and hence they can't do anything about it. Any ideas on how to fix this / how to programmatically identify drive types with 100% accuracy?

FWIW this breaks existing software like ceph that tries to detect drive types using this way. Right now the only option I have is manually fixing this. Any ideas are welcome!

Thanks. Subu

Score:1
za flag

It should. And it sets it. But Linux sets this flag based on what hardware exposes. Linux sends an SCSI INQUIRY command during hardware initialization phase to each discovered device, to know what it is, and other identification commands, and sets those flags based on what devices reply. If it replies it's a rotational HDD, it sets rotational to 1.

AWS is a virtualization cloud, so you see not a physical hardware, but virtual. It's emulated I think with some version of QEMU. You can confirm this from within a VM, if you have seen any mention of virtio devices at various places (dmesg, lspci and so on). QEMU, in turn, really has an option to expose each virtual storage device to the VM either as "rotational" or as a "solid state".

So, Linux sets whatever AWS cloud made it to set. It is the cloud duty to run QEMU with proper options. So I'd consider the reply as a formal boilerplate or run-around reply, they just don't want to investigate the problem.

rvs avatar
vn flag
rvs
It is worth noting there's unlikely a single HDD behind what AWS provides, so it won't have the same performance profile as read hardware. So treating as non-rotational could be better.
Nikita Kipriyanov avatar
za flag
This is a speculation. We just don't know and there is no way to know unless they tell us.
rvs avatar
vn flag
rvs
they don't say it directly but many things suggest that EBS works similar to things like Ceph RBD. Their FAQ directly say things like "EBS is a multi-tenant block storage service". Performance profile suggest that. And finally, as a person working on direct competitor to EBS I can say, there practically no way to build a product with similar price, durability and feature set without some sort of distributed network-attached storage underneath.
rvs avatar
vn flag
rvs
That said, I agree it's a speculation whether setting rotational to 1 is better or worse. This needs to be tested, and it may depend on specific kernel version, filesystem and workload in the guest.
Nikita Kipriyanov avatar
za flag
So again, as I said, the answer "can't do anything with it" is a run-around. They can, but don't want to do anything. The best, as I think, is to make this configurable in the VM instance control panel. Also it is worth giving the recommendation to use `noop` I/O scheduler for disks (because the real scheduling is to be performed by the host).
cn flag
So the interesting thing is - D3 instances provide 'local HDD'. These are not EBS instances. But I agree there is probably virtualization and drive sharing between various 'instances' they expose from the bare metal hardware. I will bring this back up again with AWS.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.