Score:0

Create lvm volume in the last cylinders of the disk

hm flag

In all magnetic disks, the speed difference between the first and last sectors is noticeable, up to a factor two or three. This is still true with the new multi-terabyte disks. So it still makes sense to reserve the last cylinders of the disk for infrequently used files, particularly backups.

With a partitioned disk, it is trivial to do this, with the only caveat of allowing for future changes of size.

But my configuration is unpartitioned LVM. So I need either to put two volume groups in the same physical hardware (one using the starts of a set of disks, other using the ends) or to make sure that a logical volume prefers to use the extensions in the last cylinders of the hardware. Is it possible? Do we have some control about where a LV is going to be?

Matthew Ife avatar
jo flag
That feels like a dubious claim to me in the wild. Have you actually tested how noticeable the performance actually is? By definition frequently used files will be served from page cache and be infinitely faster. On magnetic media, the fact you have to access it in the first place is a thousand times slower than retrieval from memory anyway.
hm flag
Yes, I did, for some brand new 8 Terabytes disks I was installing. I thought this was well known. You can check with a dd pipe into pv, or reading starting from different sectors.
hm flag
Now, it is true that nowadays we could consider four layers to organise the files: from memory, from ssd, from fast magnetic media, from slow magnetic media.
Matthew Ife avatar
jo flag
What I'm saying is if you perform random access on real files, they get cached to memory. The most frequent files access stay in memory. Keeping a hot pagecache greatly outperfoms any gains you'd get from physical media allocation. I'm sure theoretically you can measure a seek performance difference accessing one end of the disk to the other, yet in real-world buffered filesystems its not likely you'll encounter this type of issue as a key performance problem you'd want to tackle.
hm flag
I have some use cases in my company around pipelined scanning of huge files. Such files benefit of being in the faster stripes, and are huge enough to not be cached in memory. I am sure there are a lot of use cases in the world of big data where the files are always a miss in the pagecache.
Score:2
jo flag

I dont really agree this is a huge concern in most workloads due to caching, readahead policies and disk elevators you can use but this is possible with caveats.

The best way to tackle this would be to physically partition the media you want to allocate off into 'chunks' you consider each end of the disk you want to separate.

Something like this

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  1075MB  1074MB  primary  ext4         boot
 2      1075MB  4TB     4TB     primary               lvm # My fast partition
 3      4TB     8TB     4TB     primary               lvm # My slow partition

Then, create a volume group(s). In this example I'm using one volume group but it might be easier to have a 'slow' VG and a 'fast' VG instead..

# pvcreate /dev/sda2 
# pvcreate /dev/sda3
# vgcreate vg /dev/sda2 /dev/sda3

Then allocate your LVs out of the said physical volumes..

# lvcreate -n myFastLV -L1TB vg /dev/sda2
# lvcreate -n mySlowLV -L1TB vg /dev/sda3

Caveats here being, bad sectors can be silently replaced by the disk controller with a 'reserve' often located elsewhere (which is entirely manufacturer independent). Also some fancier disks may internally remap the sectors in that is logically consistent to the claims offered but are physically not in the place you expected them to be.

Finally, the problem workload you're suggesting (pipelining huge files) really is a very sequential workload issue that would have bigger gains from using preallocation techniques on the files written (to keep them contiguous and not fragment).

Then setting aggressive readahead policies to read whole swathes of adjacent/upcoming sectors which are most likely going to be contiguous to the file you're reading.

A finer grained approach could also be achieved using dmsetup to map the physical sectors into whatever order and fashion you wanted, but this wouldn't be too portable and probably more effort than its worth in the long run (you'd need a script to rebuild the mapping on boot for example).

hm flag
Your point about bad sectors is interesting. Of course minimising the displacement of the header is still the main trick for optimum work with magnetic disks. About readahead yes, and adequate stripes across disks, raid 0 like. Everything helps.
hm flag
dmsetup tables seems very the thing I had in my head... but if they are not saved in the LVM descriptors then it is really a headache.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.