Score:0

Ubuntu

MD5 hashing all files within a usb and saving output file on a usb

cameron.g

4/29/23, 9:56 AM

so I'm trying to work out how to hash all files within a USB directory. I came across another post similar to mine including this script below:

find "$PWD" -type d | sort | while read dir; do [ ! -f "${dir}"/@md5Sum.md5 ] && echo "Processing " "${dir}" || echo "Skipped " "${dir}" " @md5Sum.md5 already present" ; [ ! -f "${dir}"/@md5Sum.md5 ] && md5sum "${dir}"/* > "${dir}"/@md5Sum.md5 ; chmod a=r "${dir}"/@md5Sum.md5;done

It works really well and does everything I'd want it to do however I can't figure out how to alter it to target the USB. I've already got the user to input the USB device by them inputting /dev/sdf or /dev/sdg etc. But I can't figure out how to carry this information into the command above. Any advice or suggestions on this would be appreciated.

0 + 0

bash

sudodus

4/29/23, 10:12 AM

You should check that the partition on the USB drive is **[auto]mounted** (and mount it if it not). Then change directory `cd` to the mountpoint and run your script.

sudodus

4/29/23, 10:19 AM

[This link](https://askubuntu.com/questions/11840/how-do-i-use-chmod-on-an-ntfs-or-fat32-partition/956072#956072) and [this link](https://askubuntu.com/questions/958556/ubuntu-mounts-external-ntfs-drive-as-read-only/958604#958604) may be helpful if automounting of a Microsoft file system (FAT32, exFAT, NTFS) does not work.

cameron.g

4/29/23, 10:21 AM

@sudodus so it does auto mount no problem and can access everything. The command I've been writing is to clone a whole usb including a bitlocker encryption onto a identical usb. I'm wanting to create an md5 hash of the source usb which will then be copied across to the cloned drive and then do another md5 hash to check if anything has been altered during the copying.

cameron.g

4/29/23, 10:23 AM

and the reason i've got them to input /dev/sdf etc is to make sure its the whole directory of the USB not just an individual partition

sudodus

4/29/23, 10:35 AM

Well, that is different to what I thought, when you wrote 'hashing all files'. You can use `lsblk -f` and `lsblk -m` to get a list of all drives and partitions and that way identify the USB drive's device letter. You can clone it with a crude method using `dd` to another drive or to an image file, or more efficiently with [Clonezilla](https://askubuntu.com/questions/958242/fastest-way-to-copy-hdd/958248#958248). It is difficult to get a reproducible md5sum of the cloned drive because the target drive might be bigger than the original one. Instead **test that the cloned copy works correctly**.

cameron.g

4/29/23, 10:41 AM

I use dd as that appeared to be quickest. Clonezilla didn't work correctly for me for whatever reason and wasn't really user friendly... I've got everything sorted within my command and the end users know how to find the usb device name. I've been making clones for the past month or so using my command so i know it works. It's just purely trying to get this command I originally commented about to work with what i have.

sudodus

4/29/23, 10:54 AM

Are you making an image file or a cloned USB drive copy?

cameron.g

4/29/23, 10:56 AM

A bit for bit copy of the USB

sudodus

4/29/23, 11:06 AM

Then you should be able to printout the used size of the source (original) USB drive (to the end of the last partition). Then run `dd` using bs and count and pipe the result to `md5sum`. After cloning you should check that you get the same result on the target drive. Please notice that drives are usually not exactly the same size (in bytes), so it is important to check with the same numbers for bs and count. (The target drive must be big enough to copy until the end of the last partition of the source drive.)

cameron.g

4/29/23, 12:04 PM

So I haven't tried what you've suggested yet (@sudodus). But I tried altering the above to this: `find "/dev/$source" -type d | sort | while read source; do [ ! -f "${source}"/@md5Sum.md5 ] && echo "Processing " "${source}" || echo "Skipped " "${source}" " @md5Sum.md5 already present" ; [ ! -f "${source}"/@md5Sum.md5 ] && md5sum "${source}"/* > "${dir}"/@md5Sum.md5 ; chmod a=r "${source}"/@md5Sum.md5;done` After doing this, no progress is appeared to made and nothing is created in said USB... Soooo im a little bit stumped now

sudodus

4/29/23, 5:23 PM

Instead of trying to modify your script, I wrote an answer where I focus on what I think are important steps in order to check the 'whole' USB drive. I hope it will help you to reach your goal.

Score:0

Ubuntu

sudodus

4/29/23, 4:20 PM

I made a test with the Ubuntu Server iso file ubuntu-20.04.3-live-server-amd64.iso

from https://releases.ubuntu.com/20.04.3/ with the sha256sum

f8e3086f3cea0fb3fefb29937ab5ed9d19e767079633960ccb50e76153effc98

The iso file was cloned to USB drive. (You can do it with any cloning tool.)
The used size on the USB drive was determined with
```
sudo fdisk -lu /dev/sdx
```
where x is the drive letter (identified with fdisk and for example lsblk -f) and the number of sectors (from 0 to 'end_sector') is end_sector + 1, so in my test example 2463616 and the sector size is 512 bytes.
Check that no file system in the drive is mounted, and unmount the partition(s) if mounted (but do not eject).
Check on the USB drive

Calculate a checksum on the whole used part of the USB drive,
```
sudo dd if=/dev/sdx bs=512 count=2463616|sha256sum
```
The output of this check is the same as the uploaded checksum, which verifies that the cloning operation was successful. You may prefer to pipe the output from dd to md5sum.
If the checksum does not match for you, please check first that the number of sectors and number of bytes are correct (that you have checked exactly the correct part of the USB drive).

This same method works also when you clone from one USB drive to a another drive that is big enough, so check on the new drive (sha256sum or md5sum according to your choice).

0 + 0

cameron.g

4/29/23, 5:10 PM

Thanks for this so much sudo.... Lastly I'd prefer a txt file containing the md5 inside of it which i can create. But it always saves to my desktop how can i save it to other places like my usb?

sudodus

4/29/23, 5:41 PM

You can save the result to a file in a mounted file system, but not into the drive that you are checking. In order to avoid problems, you should also check that no file system in the drive to check is mounted, so unmount if necessary (but do not eject because that turns off the power to the drive). So use the full path `/mountpoint/directory-path/file-name` to where you want to save the result.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: MD5 hashing all files within a usb and saving output file on a usb

TH: MD5 แฮชไฟล์ทั้งหมดภายใน usb และบันทึกไฟล์เอาต์พุตบน usb

RO: MD5 hashing toate fișierele dintr-un USB și salvează fișierul de ieșire pe un USB

RU: MD5 хэширует все файлы на USB-накопителе и сохраняет выходной файл на USB-накопителе.

VI: MD5 băm tất cả các tệp trong usb và lưu tệp đầu ra trên usb

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.