Score:0

Why is mv so slow - local disk to local disk?

st flag

I'm moving a single large file from /mnt/z to /mnt/y on a bare metal Ubuntu 20.04.3 install.

Both disks are capable of sustaining 2 GB/s in writes, and do in other operations, but mv caps out at ~200 MiB/s.

The disks are PCIe 4.0 Highpoint RAID cards with either U2 or M2 drives in RAID 0. Both with NTFS partitions.

What could I be missing?

ar flag
Are these USB 2 or 3.1 or 3.2 drives or external SATA drives? If these are SATA drives in USB cages, is [UASP](https://www.minitool.com/lib/uasp.html) working? Please edit your question and add the new information.
st flag
Neither. They're in the machine, directly attached via PCIe lanes.
vanadium avatar
cn flag
Perhaps because the file system format is ntfs? Would the performance not be better when using a native linux file system like ext4, or perhaps with vfat? Is it also so slow when copying many smaller files?
sudodus avatar
jp flag
+1 to @vanadium's comment. My experience is the NTFS is much slower than ext4 when accessed from Linux.
cn flag
@vanadium 99% sure that is the issue -- Move them using windows otherwise NTFS is not the best filesystem for this; You can use ext3 for 100% OS support. ext4 might need a 3rd party driver on some other OSes
cc flag
Search this site for "slow copy". e.g. see https://askubuntu.com/questions/1113235/very-slow-copying-files-from-exfat-to-ntfs In a nutshell, the copy process hands off the output to the system (buffers), system buffers fill up, things drag to a crawl. Happens any time the output is slower than the input, and the copy is big enough to fill the systembuffers (memory dependent).
st flag
It's interesting that writing to either NTFS disk from memory goes at full speed. I can also copy files to different disks simultaneously and the total read from the NTFS disk rises, but each individual mv operation still caps out around 200MiB/s.
st flag
@ubfan1, would this be relevant with a 100GB file and 500GB RAM? Both the read and write are happening at 1/10th the bandwidth.
cc flag
Please add additional info to your posting instead of in comments. No experience with 500GB memory, but I see slowdowns when my copy size exceeds my memory. Bad news is the SMR hardware that is just slow -- honest vendors state "not suitable for backups".
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.