dd is vastly slower than gparted to copy partitions. How is that possible?

Dov

7/21/24, 7:35 PM

I tried to copy partitions from a 500GB internal M.2 SSD to an external USB3.x enclosure around an M.2 2TB drive.

My first attempt used an A to C cable, and the command:

dd if=/dev/nvme01p3 of=/dev/sda1 bs=1M

took approximately 6.5 hours for a 240GB partition. It was reporting 10MB/s. This seemed really slow. It turned out that the port I was connecting on was probably USB2, so I used the short USB C to C cable, and got 40MB/s, and it took about 1.5 hours for a similar partition.

dd should be pretty low level, particularly if the files in question are partitions. Isn't that just raw block on the disk? So when in another question, some answers suggested I use copy partition in gparted. There were mistakes anyway, so I tried that, and was stunned when it achieved what I thought it should, which was more than 1GB/sec. 4 minutes later, the partition was redone.

The question is, how is it possible that dd can be 40x slower than gparted? If this question has to move to stackoverflow, I will do it, but it's in a gray area between software and linux.

283

2 + 0

gparted

Score:3

Ubuntu

Rinzwind

7/21/24, 7:50 PM

'dd's speed depends on the block size.

Block sizes/buffer sizes/read ahead settings all have an impact on a sequential read/write. If you use small block sizes, the drive will only read small amounts of data per access before passing it on to the next process.

Find the sweet spot and dd will be as quick as gparted.

Gparted will have a method to determine that sweet spot. Probably sudo blockdev --getsz /dev/{device}.

+ 1

Dov

7/22/24, 5:13 AM

I don't believe that block sizes of 64k and 1M could possibly be that bad. We are talking about a factor of 40 speed increase.

Score:3

Ubuntu

Raffa

7/23/24, 1:28 PM

dd should be pretty low level, particularly if the files in question are partitions. Isn't that just raw block on the disk?

True, but raw blocks are the same for used(occupied by data) and unused(empty/all zeroes/sparse/filesystem with no data i.e. files) ... So, as far as dd is concerned, those are copied on the disk raw disk block level bit by bit overlooking their contents ... i.e. same speed for all disk blocks.

Gparted on the other hand is a filesystem aware utility ... To know how that might affect partition(filesystem) copying speed, try copying an empty partition with only a filesystem i.e. just formatted and no files stored on it ... dd will copy each individual block i.e. as if the filesystem is full of files which might take a long time depending on the size of the source and the speed of both disks and the link between them, while gparted on the other hand will just recreate the empty filesystem on the target free space and that should be blazingly fast as it's merely formatting the target space with identical size and same type filesystem.

As it seems, your source partition/filesystem is not occupied with enough files/data to force gparted to copy every single block's contents.

Filesystem aware utilities will be faster when the source partition is not full and has a lot of free space, while dd might be faster or of at least same speed otherwise.

Please see a mostly related extended discussion to this point in copying 500 Gig from one portable to another using dd

+ 0

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: dd is vastly slower than gparted to copy partitions. How is that possible?

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.