Score:2

How to bind spdk driver to NVMe device if root filesystem is mounted there?

cn flag

I am trying to use SPDK in Ubuntu 20.04 (Linux kernel 5.4.0) to run storage applications. My NVMe SSD is allocated as below:

$ lsblk
...
nvme0n1                   259:1    0   1.5T  0 disk
├─nvme0n1p1               259:2    0   512M  0 part /boot/efi
├─nvme0n1p2               259:3    0     1G  0 part /boot
└─nvme0n1p3               259:4    0   1.5T  0 part
  └─ubuntu--vg-ubuntu--lv 253:0    0   200G  0 lvm  /

I followed the getting started guide from SPDK docs. It is suggested that any filesystem on the device should be unmounted before binding SPDK driver. Since my root (/) filesystem is mounted in the device, I ran umount / before running scripts/setup.sh. It's likely that the SPDK driver does not bind to the NVMe device properly as the example applications like identify do not run. I am not sure if the filesystem is unmounted properly as I still use the shell (/bin/sh) after that.

When I try to go back to Linux native kernel driver with scripts/setup.sh reset, the nvme device gets a new name nvme0n2 and the whole filesystem remains read-only. I have to then physically restart the machine again.

How can I use SPDK in this case?

Score:3
kz flag

You cannot do that. Get second NVMe just for test & development.

Score:-1
cn flag

Got answer from SPDK Slack channel that I cannot bind the SPDK driver to a device with the root filesystem mounted. I either need to have a separate device to contain the root filesystem or use a VM to run SPDK.

I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.