Efficient way to write a sparse partition to a disk image?

Fadeway

3/10/24, 1:27 PM

I generate an ext4 partition image that contains a minimal Debian file system, then write that partition image into a full disk image. I am looking for the fastest way to do the write.

Consider a minimal test case:

truncate -s 30GB disk.img
parted -s disk.img mklabel gpt
parted -s disk.img mkpart small ext4 1MiB 301MiB
parted -s disk.img mkpart root ext4 301MiB 100%

truncate -s 20GB rootfs.img
truncate -s 300MB small.img
mkfs.ext4 rootfs.img
mkfs.ext4 small.img

# In reality, here we might put something on the partitions, like:
# mount rootfs.img /mnt/root; debootstrap /mnt/root

# The part we want to optimize:
dd if=small.img of=disk.img seek=1 bs=1M conv=notrunc
dd if=rootfs.img of=disk.img seek=301 bs=1M conv=notrunc

The rootfs.img and small.img files generated in the code above contain just an empty ext4 filesystem and are thus very small (~67MB for a 5G filesystem, 120MB for a 30G). Despite this, dd takes a long time to copy them over, even with conv=sparse. Presumably this is because dd looks at the full 20GB and not just the used area as printed by du -sh. Plus, using conv=sparse is a bug if the target is not zeroed out.

What is the fastest way to copy a sparse partition image to the middle of a disk image? An ext4-only solution is sufficient, but filesystem-independent methods are even better.

422

1 + 1

linux

partition

disk-image

paladin

3/16/24, 10:05 AM

I would use `resize2fs -M small.img && resize2fs -M rootfs.img`, after the filesystems are filled with all needed data.

Score:3

Server

Jasen

3/10/24, 9:05 PM

Use losetup to create loop devices that point to the partitions in the large image file.

losetup -P loop9 disk.img

creates

/dev/loop9 representing the whole file
/dev/loop9p1 representing the first partition
/dev/loop9p2 representing the second partition

Then use partclone.ext4 to copy the data from the small image files to the loopback devices. (for the stretch goal other flavours of partclone can be used for other filesystems)

Or you could instead just format the loopback devices, mount them and install the files you want directly.

+ 0

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Efficient way to write a sparse partition to a disk image?

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.