pacemaker agent nvmet-subsystem missing

kumpel4

8/12/24, 5:07 PM

I tried to follow the drbd Howto "Highly Available NVMe Over Fabrics (NVMe-oF) on RHEL 9", but run into the error

`Error: Unable to find agent 'nvmet-subsystem', try specifying its full name`
`Error: Errors have occurred, therefore pcs is unable to continue`

It happend with the command:
pcs -f nvmet_config resource create p_nvme-subsystem_0 nvmet-subsystem \ nqn=linbit-nqn0 \ op start timeout=10s interval=0 \ op stop timeout=10s interval=0 \ op monitor interval=10s timeout=10s

So I have no experience with pacemaker, but i think it requests the NVMe-oF target export agent, that is located in the package "resource-agents". So I installed it additionally although it was not specified. But no change.
I found this overview according agents:
pcs example for nvme-subsystem
So I changed the command to:
pcs -f nvmet_config resource create nvme-subsystem_0 ocf:heartbeat:nvmet-subsystem nqn=drbd-nvme1 op......
Also not working. I tried this Howto with Rocky 9, Ubuntu Server 22.04 and Debian 11.3. All give the same error message. Clear is that the Howto is not correct, but how can i fix this? I found not one post in the internet according this. Strange. I dont think that i am the first, because Linbit do it already.

225

1 + 0

pacemaker

nvme

Score:3

Server

BaronSamedi1958

8/13/24, 7:46 PM

There’s no point to proceed this route either way as DRBD kills NVMe performance during replication. You need some 100% user-mode application doing polling and avoiding interrupt processing overhead, thread context switches, and user-to-kernel (and back) jumps. Nothing living in kernel would fully unleash NVMe potential. Say SPDK NVMe-oF target might be a way better choice.

https://spdk.io/releases/

EDIT: Replaced commercial Lightbits with open-source SPDK.

+ 5

kumpel4

8/13/24, 8:11 PM

no advertising, please

BaronSamedi1958

8/14/24, 7:13 AM

I don’t have anything to do with the Lightbits guys. Replaced the link with an open-source SPDK though.

kumpel4

8/14/24, 10:50 AM

Nice to Know there exist a opensource Framework. The performance charts are impressiv. I like that there is support for compression on blocklevel. For my understanding SPDK uses nvme internally so u can not use normal SAS-HDDs. Right?

kumpel4

8/14/24, 1:57 PM

That was not formulated correctly. I mean SPDK uses the nvme-protocol for "disks".

BaronSamedi1958

8/15/24, 7:52 AM

You can absolutely attach to the aio-exposed Linux kernel block devices. We’ve did it even with a spinning disks (just don’t ask why…), so kernel-managed NVMe shouldn’t be an issue. Just make sure there’s no root file system mounted on top of it.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: pacemaker agent nvmet-subsystem missing

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.