Score:1

Slow NVMe speed on new install

ca flag

I've just setup an HP z230 Xeon E3 1225 v3 machine with an HP TurboDrive G1, with a Seagate Firecuda NVMe SSD. I was expecting 2-3000MB/s read speeds, but am seeing way less than that. I don't care too much about a few hundred here and there, but I'm seeing 300-400MB/s, which makes me think something is misconfigured.

$ uname -ra
Linux host 5.4.0-74-generic #83-Ubuntu SMP Sat May 8 02:35:39 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.2 LTS
Release:    20.04
Codename:   focal

Simple partition scheme, no encryption

Disk /dev/nvme0n1: 931.53 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: Seagate FireCuda 510 SSD ZP1000GM30031
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 521B5816-364D-4D9B-B9B5-5F7C17E272C9

Device         Start        End    Sectors   Size Type
/dev/nvme0n1p1  2048 1953521663 1953519616 931.5G Linux filesystem

hdparm showing nearly an order of magnitude below expectations:

$ hdparm -tT --direct /dev/nvme0n1

/dev/nvme0n1:
 Timing O_DIRECT cached reads:   722 MB in  2.00 seconds = 360.94 MB/sec
 HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device
 Timing O_DIRECT disk reads: 1098 MB in  3.00 seconds = 365.60 MB/sec

Playing with grub settings:

GRUB_CMDLINE_LINUX_DEFAULT="quiet splash nomodeset nvme_core.default_ps_max_latency_us=200 scsi_mod.use_blk_mq=1"

Scheduler

$ cat /sys/block/nvme0n1/queue/scheduler
[none] mq-deadline

fstab:

/dev/disk/by-uuid/26ccd344-b8f1-4101-af69-6571bb600a87 / ext4 defaults,noatime,nodiratime 0 0

smartctl:

smartctl --all /dev/nvme0n1
smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.4.0-74-generic] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       Seagate FireCuda 510 SSD ZP1000GM30031
Serial Number:                      7W1004GW
Firmware Version:                   STPSC014
PCI Vendor/Subsystem ID:            0x1bb1
IEEE OUI Identifier:                0x6479a7
Total NVM Capacity:                 1,000,204,886,016 [1.00 TB]
Unallocated NVM Capacity:           0
Controller ID:                      1
Number of Namespaces:               1
Namespace 1 Size/Capacity:          1,000,204,886,016 [1.00 TB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            6479a7 486f001b5c
Local Time is:                      Mon Jun 14 20:17:02 2021 UTC
Firmware Updates (0x1e):            7 Slots, no Reset required
Optional Admin Commands (0x0017):   Security Format Frmw_DL Self_Test
Optional NVM Commands (0x005d):     Comp DS_Mngmt Wr_Zero Sav/Sel_Feat Timestmp
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     75 Celsius
Critical Comp. Temp. Threshold:     80 Celsius

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     9.48W       -        -    0  0  0  0        0       0
 1 +     6.45W       -        -    1  1  1  1        0       0
 2 +     4.93W       -        -    2  2  2  2        0       0
 3 -   0.0490W       -        -    3  3  3  3     2000    2000
 4 -   0.0018W       -        -    4  4  4  4    25000   25000

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         2
 1 -    4096       0         1

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        35 Celsius
Available Spare:                    100%
Available Spare Threshold:          5%
Percentage Used:                    0%
Data Units Read:                    206,765 [105 GB]
Data Units Written:                 209,598 [107 GB]
Host Read Commands:                 14,114,924
Host Write Commands:                11,352,976
Controller Busy Time:               11
Power Cycles:                       25
Power On Hours:                     34
Unsafe Shutdowns:                   20
Media and Data Integrity Errors:    0
Error Information Log Entries:      6
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0

Error Information (NVMe Log 0x01, max 63 entries)
No Errors Logged

Alignment:

$ sudo parted /dev/nvme0n1
GNU Parted 3.3
Using /dev/nvme0n1
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) align-check opt 1
1 aligned

And thus concludes my experiments so far. I'm keen to find other things to try, or if its posible something is up with the SSD. (though it is brand new)

Thanks, Chris

oldfred avatar
cn flag
Have you updated firmware for UEFI and SSDs? That sometimes helps. I also prefer to use a smaller / of 30GB and then large /home and/or data partition(s). The nomodeset boot parameter is for video issues, until you install correct video driver. To see firmware version (F/w revision): `sudo apt install nvme-cli` & `sudo nvme list` & `sudo nvme --help`
Chris McV avatar
ca flag
thanks @oldfred - UEFI firmware was updated to latest, and there isn't any available for the SSD yet. I assume the 30GB os partition is a style preference and not a factor in the performance?
oldfred avatar
cn flag
Not performance, at least with newer SSD. Maybe back with HDD. But it makes backup & new install or repair install somewhat easier. I like to have multiple installs, so when testing some major configuration change, I use another install, not main working install. And then I mount my data partitions, and keep /home inside /. I also move Firefox & Thunderbird profiles to data partition, so everything is in test install except user configuration. But for first time in 15 years had issue with FF version, not wanting to revert to old version after using new.
Crouching Kitten avatar
ro flag
If the NVMe is on a PCIe x16 slot, then for a single thread of read/write operation, it will use 1/16 of the full speed. It will run on full speed only if 16 thread is used, or if the program accesses multiple blocks in parallel in an async way. I'm pretty sure this could be solved on the kernel level, as Windows does parallel block read/write too for a single thread automatically.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.