Join Log in
Score:1
Marwi avatar Server

Disk performance issues exclusively with RAID 10 (LVM RAID)

fr flag
Marwi

For some reason, I am suffering very poor disk performance on RAID 10, but normal performance on RAID 5.

For context, I have 4 x 1 TB hard drives in a server. They are the physical volumes within the volume group.

  PV         VG Fmt  Attr PSize    PFree
  /dev/sda1  vg lvm2 a--  <931.51g    0 
  /dev/sdb1  vg lvm2 a--  <931.51g    0 
  /dev/sdc1  vg lvm2 a--  <931.51g    0 
  /dev/sdd1  vg lvm2 a--  <931.51g    0

I created one RAID 10 logical volume with size 30 GB for the OS (root), and another RAID 5 logical volume with the rest of the free space for storing user data. Both span the whole 4 disks.

  • lvcreate --type raid10 --mirrors 1 --stripes 2 -L 30G -n root vg
  • lvcreate --type raid5 --stripes 3 -l 100%FREE -n data vg

When installing Debian on the root LV, the installer took more than two hours to complete. I knew something was wrong, so after the installation was complete, I ran a simple dd disk benchmark on the root LV, and got less than 10 Mb/s for the write performance. I then ran the same benchmark on the data LV, and got around 150 Mb/s.

As observed, the RAID 5 data LV is much faster, even though the RAID 10 root LV should outperform it in this case.

To confirm these benchmarks, I repeated the exact same install but on the data LV this time, and the installation took only 15 minutes.

Why is there such poor disk performance when using RAID 10? I have provided information about the CPU I'm using, if that's any help.

# dmidecode 3.3
Getting SMBIOS data from sysfs.
SMBIOS 2.5 present.

Handle 0x0004, DMI type 4, 40 bytes
Processor Information
        Socket Designation: CPU 1
        Type: Central Processor
        Family: Unknown
        Manufacturer: Intel            
        ID: 7A 06 01 00 FF FB EB BF
        Version: Pentium(R) Dual-Core  CPU      E5200  @ 2.50GHz     
        Voltage: 1.2 V
        External Clock: 200 MHz
        Max Speed: 2500 MHz
        Current Speed: 2500 MHz
        Status: Populated, Enabled
        Upgrade: Socket LGA775
        L1 Cache Handle: 0x0005
        L2 Cache Handle: 0x0006
        L3 Cache Handle: 0x0007
        Serial Number: To Be Filled By O.E.M.
        Asset Tag: To Be Filled By O.E.M.
        Part Number: To Be Filled By O.E.M.
        Core Count: 2
        Core Enabled: 2
        Thread Count: 2
        Characteristics:
                64-bit capable
326
0 + 2
lvm
Zareh Kasparian avatar
us flag
Zareh Kasparian
I had the same issue, please refer to the link below and test your case with mdadm rather LVM. https://recoverhdd.com/blog/comparison-and-difference-between-raid-lvm-and-mdadm.html
0
Reply
Marwi avatar
fr flag
Marwi
@ZarehKasparian nope, the issue persists even on mdadm RAID. I discovered that it has nothing to do with the RAID type at all, but that the disks were still synchronizing from the previous install, which causes subsequent read and writes to be much slower until it finished synchronizing. (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=838503)
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.