Using dd with conv=sparse for creating partition or full disk images?

user79

2/14/23, 2:46 AM

For some reason there don't seem to be too many questions or answers about using dd for creating "smaller" image files with the conv=sparse option here.

So when using dd for image creation, can I safely use e.g. dd if=/dev/sda of=/media/storage-device/sda.img conv=sparse to get an image file that uses (much) less space on the target?

Don't know if it matters, but in this case the source would be a SSD with EXT4 filesystem and only 15% used (as it is quite new), and the target also would use EXT4 fs.

Or is this possibly a rather bad idea (under certain circumstances) and may lead to problems (later, e.g. when I need to restore it)?

I am also wondering if this would work when the target is a network mount or filesystem?

The goal would be to get an image file of either a partition or (even better) a whole disk that doesn't take much more space than needed, while keeping it easily mount- and usable.

Extra compression isn't really needed or wanted, and it usually also would make the resulting image not mountable any more I guess, so I can live without it.

And I know Clonezilla can do stuff like saving only used sectors, but I don't like its limitations, e.g. Due to the image format limitation, the image can not be explored or mounted. (And of course dd is already available on every system.)

Many thanks in advance

322

0 + 0

partitioning

disk-image

muru

2/14/23, 2:51 AM

If the disk usage is really that low, even simple `gzip` compression will likely give a small image and that is definitely safe.

sudodus

2/14/23, 8:27 AM

Compression or using things like conv=sparse will only be efficient, if you zeroise the free space before. It can be done with `zerofree` with ext file systems or generally with dd by creating a huge file filled with zero-bytes and then remove that file. This is a slow process, so I prefer Clonezilla, but I understand that you want to explore this process. Anyway, I don't know if the method you suggest will work, and I'm looking forward to your result :-P

sudodus

2/14/23, 8:38 AM

Otherwise, an obvious alternative is to work on the file level, use `tar` with or without compression to get a useful 'image' of a partition. This is not a true image, but can be very useful in order to store the content, so that it can be restored to the same and even to a smaller partition.

user79

2/14/23, 2:21 PM

I see that every approach has some advantage (but also downsides) Probably I could pipe the output of dd to gzip or so, and so the target filesystem doesn't even have to support sparse files and the resulting image file may even be smaller - but in order to be able to mount it or restore single files from it, I would first have to extract it somewhere with _plenty_ of free disk space. And if the archive's header gets corrupted for some reason, it will make the whole image entirely useless because I couldn't uncompress it any more I guess. So one should probably store a 2nd image somewhere else

user79

2/14/23, 2:51 PM

"will only be efficient if you zeroise the free space before" - In this case this shouldn't be neccessary as the drive is still pretty new, but generally speaking you're right, so thanks for the info. I just thought this kind of would be done automatically on SSDs by trim command...? - I probably will end up creating one "sparse" image, a somehow compressed one, and maybe a tar archive, just in case one file gets corrupted or can't be restored correctly for some reason :D

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Using dd with conv=sparse for creating partition or full disk images?

TH: ใช้ dd กับ conv=sparse เพื่อสร้างพาร์ติชันหรืออิมเมจดิสก์เต็มหรือไม่

RO: Folosind dd cu conv=sparse pentru a crea partiții sau imagini de disc complet?

RU: Использование dd с conv=sparse для создания разделов или полных образов дисков?

VI: Sử dụng dd với conv=sparse để tạo phân vùng hoặc hình ảnh toàn bộ đĩa?

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.