Validating an ISO write directly after writing

guiverc

5/14/24, 4:34 AM

Background

My normal process for writing ISOs is downloading them (zsync via script which performs a ISO validation & shows me the package changes (ie. manifest diff from prior download)), then use mkusb to write it to thumb-drive, eg.

guiverc@d7050-next:/de2900/lubuntu_64$ sudo mkusb-nox mantic-desktop-amd64.iso all

I then quickly scan messages & if it looks good, eject the thumb-drive, and boot it on a nearby box & confirm media write there using a method documented here, ie.

boot it in try mode, and wait awhile to let background media validation occur, scanning the systemd journal using

sudo journalctl |grep casper-md5check

watching for

May 11 08:37:47 ubuntu casper-md5check[3924]: Checking ./boot/grub/loopback.cfg...../boot/grub/loopback.cfg: OK
May 11 08:37:47 ubuntu casper-md5check[3924]: Check finished: no errors found.
May 11 08:37:47 ubuntu systemd[1]: Finished casper-md5check.service - casper-md5check Verify Live ISO checksums.

This validation I believe is essential, as I find 5-8% of ISO writes fail regardless of the hardware being used (ie. different boxes & different thumb-drives).

Question

Can I validate the ISO write using an easy tool, using the same box I used to write it (ie. no reboot or nearby box necessary).

My preference is terminal; as it's where I am for these tasks.

1 + 1

iso

guiverc

5/14/24, 4:36 AM

Likely obvious.. but this is an *engineered* question, asked to help publish a new tool... All answers of course are welcome, and sorry if it's a *duplicate* (*I didn't find a question, but I may have used the wrong terms!*). @sudodus

Score:2

Ubuntu

sudodus

5/14/24, 3:47 PM

Tool to check that cloning was successful

Cloning from an image file, typically an iso file, to a USB drive is considered a robust process, but sometimes it fails. Maybe this is caused by problems with buffering when a big iso file is cloned to a pendrive with slow memory cells, maybe caused by flaky memory cells.

Anyway, I see that we need a convenient way to check that cloning was successful, so I created a small shellscript.

diff-image-drive makes it easy to check by means of comparing the sha256sums.

Demo using the original version of the shellscript:

At first there is some previous content in the target device, an SSD connected via a USB to SATA adapter:

$ sudo diff-image-drive mantic-desktop-amd64.iso /dev/sdc
 Comparing content of 'mantic-desktop-amd64.iso' and '/dev/sdc' 
partprobe: 
umount /dev/sdc*: 
done - no partition mounted on /dev/sdc 
 calculating sha256sums ...
2,92GiB 0:00:18 [ 163MiB/s] [================================================================>] 100%            
file:   cdfaafb6be7b4b183b43bf395f7a1e12b14bcc6914c13ca8246a4cb00254ef6e  -
2,92GiB 0:00:14 [ 203MiB/s] [================================================================>] 100%            
device: ac65f5bb93207976cf59e4440147ffdda40e6e0ac3a3f964f525c0063660b51d  -
 content of iso/img file and device differ

After cloning from the mantic iso file to the target device:

$ sudo diff-image-drive mantic-desktop-amd64.iso /dev/sdc
 Comparing content of 'mantic-desktop-amd64.iso' and '/dev/sdc' 
partprobe: 
Warning: Not all of the space available to /dev/sdc appears to be used, you can fix the GPT to use all of the space (an extra 462737388 blocks) or continue with the current setting? 
umount /dev/sdc*: 
done - no partition mounted on /dev/sdc 
 calculating sha256sums ...
2,92GiB 0:00:14 [ 208MiB/s] [================================================================>] 100%            
file:   cdfaafb6be7b4b183b43bf395f7a1e12b14bcc6914c13ca8246a4cb00254ef6e  -
2,92GiB 0:00:14 [ 204MiB/s] [================================================================>] 100%            
device: cdfaafb6be7b4b183b43bf395f7a1e12b14bcc6914c13ca8246a4cb00254ef6e  -
 content of iso/img file and device match

In this case you need not worry about the warning from partprobe. A drive cloned from an iso file does not care about what is behind the cloned field.

Screenshots using version 1.3 of the shellscript:

The color high-lighting is there also in the original version, but checking when skipping head/tail is new. This helps checking when a drive cloned from an Ubuntu iso file has been used (so that a 'writable' partition was created).

Checking directly after cloning.

Checking after the drive was booted into.

You can find more details via this post and the following posts to my thread at the Ubuntu Forums.

+ 0

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Validating an ISO write directly after writing

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.