OS details:
PRETTY_NAME="Ubuntu 22.04.2 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.2 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
I had a ceph cluster based on ubuntu LTS, for proxmox storage using rbd, but vm disk speed (no matter read or write) only lower than 10MB/s, somethimes KB/s, this is background.
So I made a test env for find out why so slow, 10GB switch, 10GB netcard, 2 node, one for ceph(ubuntu 22.04, name ceph node), one for just ubuntu 22.04(name client node)
6 disk in ceph node( 3 ssd, 3 hdd), problem is I tested for ssd and hdd, almost same speed, so I tried to do everything in ceph node so I can skip the network
test prograss:
- ceph node map(127.0.0.1) rbd from ceph node(127.0.0.1) - sudo rbd map speedtest/aaa --id admin --keyring /etc/ceph/ceph.client.admin.keyring --mon-host 127.0.0.1
 
- fio to test, 2226KiB/s for write speed 
    direct=1
    rw=write
    filename=/dev/rbd0
    [job]
    name=test
    bs=4k
    iodepth=16
    size=200G
    runtime=60
    numjobs=1
    time_based
    rwmixread=70
- than I remove all osd, direct to test disk /dev/sdg, 203MB/s for write speed
    [global]
    direct=1
    rw=write
    filename=/dev/sdg
    [job]
    name=test
    bs=4k
    iodepth=16
    size=200G
    runtime=60
    numjobs=1
    time_based
    rwmixread=70
- I guess what if ceph configuration has something wrong, so I create a nfs server for localhost, write speed but still slowwwwwwwwwww, 1028kB/s - my /etc/exportfs: -     /media/speedtest *(rw,sync,all_squash,anonuid=0)
 - command: -      sudo mount.nfs localhost:/media/speedtest /mnt/
 - mount disk: -      /dev/sdd1 on /media type ext4 (rw,relatime)
 - fio config same as above 
- I tested for dir /media, write speed 199MB/s 
enter image description here
enter image description here
I almost gonna be crazy, how is this happend ? somebody help me please, if you need something else information, tell me, I will show you everything if you need.
help me, please.