Score:1

Progress bar / Loading icon whilst creating an MD5 hash for a USB

cm flag

I'm trying to create a progress bar or some sort of loading icon to show that a hash is being created, instead of a blank screen with nothing happening... This is what I have so far:

if [[ $hashing != "y" && $hashing != "Y" ]]; then
                        echo -e "\n"
                        sudo dd if=dev/"$source" | md5sum
                                read -r compareresult
                                        i=1
                                                sp="/-\|"
                                                echo -n ' '
                                                while true
                                                do
                                                        printf "\b${sp:i++%${#sp}:1}"
                                                done
                                        exit        
                    fi ;;

This uses a spinning wheel to represent a loading icon. And it does work using what's above. But doesn't at the moment? Not sure if the order is incorrect or if I'm missing something. Any help would be great thanks

#!/bin/bash
#Clone_Command
while true
    do
    sudo -s
    echo "==========================="
    echo "   Clone Command    "
    echo "==========================="
    echo -e "\n"
    
    echo -e "\n"
    
    echo "Enter 1 for source device"
    echo "Enter 'a' to hash source device"
    echo "Enter 2 for destination device"
    echo "Enter 3 to list all available disks"
    echo "Enter 4 to execute dd command"
    echo "Enter 5 to compare MD5 hashes"
    echo "Enter q to exit"
    echo -e "\n"
    echo "PLEASE NOTE LISTING ALL DISKS WILL REQUIRE YOU TO RELOAD THE SCRIPT"
    echo -e "\n"
    echo -e "Enter your choice \c"
    read -r choice
    case "$choice" in
        q) exit;;
        1) echo -e "Enter source device '/dev/---'
                
Enter the last 3 letters of the device eg - sdf or sdb etc"

            read -r source ;;
            
        a) echo -e "Hashing this device may take a while depending on size"
                echo -e "\n"
                echo -e "Press enter if you wish to hash this device"
                
                read -r hashing 
                
                        if [[ $hashing != "y" && $hashing != "Y" ]]; then
                        echo -e "\n"
                        sudo dd if=dev/"$source" | md5sum
                                read -r compareresult
                                        i=1
                                                sp="/-\|"
                                                echo -n ' '
                                                while true
                                                do
                                                        printf "\b${sp:i++%${#sp}:1}"
                                                done
                                        exit        
                    fi ;;
            
        2) echo -e "Enter destination device '/dev/---'
                
Enter the last 3 letters of the device eg - sdf or sdb etc"

            read -r destination ;;
            
        3) echo -e "Press enter to list all available disks \c"
            read -r answ
        if [[ $answ != "y" && $answ != "Y" ]]; then
        clear
        sudo lshw -class disk
        exit
        fi ;;
        
        4) echo -e "This will format $destination. If you wish to continue press enter \c"  
            read -r ans
        if [[ $ans != "y" && $ans != "Y" ]]; then
        echo -e "\n"
            sudo dd if=/dev/"$source" of=/dev/"$destination" bs=4096 status=progress
            exit
        fi ;;
        
        5) echo -e "If you wish to compare MD5 Hash of both USBs then press 'Enter'\c"
                read -r compare
        if [[ $compare != "y" && $compare != "Y" ]]; then
               echo -e "\n"
               echo -e "Please note this is not a quick process"
               echo -e "n"
               md5sum -c <<<"$compareresult  /dev/$destination"
        fi ;;
    esac
done            
sudodus avatar
jp flag
What software or data is there on the USB drive? Did you make it yourself? Or do you want to check something that you installed from an image file or iso file? What do you plan to read into `compareresult` ? -- The spinner starts *after* md5sum has finished. I suggest to put it into the background (with `&`) and check for the process to finish or put the spinner into the background. -- An alternative is to create an image file (maybe that's what you want anyway) and then run `md5sum` on that file. -- Another alternaitve is to use `pv` (progress view) instead of `dd`.
cm flag
@sudodus The USB stick is made up of pdfs, word docs etc. With a bitlocker encryption on it as well. I was hoping `compareresult` would take the created md5 hash of the whole USB to verify against the cloned USB stick, which I don't think it doesn't do lol. With regards to putting the spinner in the background where would I add this? I'll add the whole script as well to make it easier to understand.
bac0n avatar
cn flag
You have to get your tabs and space in order it looks just too miserable.
Score:0
jp flag

Progress view with pv

There is a general problem with your concept because USB sticks have slightly different sizes even when the nominal size is the same, and that will affect the md5sum of the whole device.

  • If you check the md5sum of each partition, you can check that the original one and the cloned copy are the same.
  • An alternative is to store an image file and use the md5sum and the size of the image to apply on the sticks (checking with dd exactly the same number of bytes as in the image file).

Instead of a spinner you can use pv, progress view, for example

$ sudo pv /dev/sdc | md5sum
29,8GiB 0:13:46 [37,0MiB/s] [================================================>] 100%            
2372da0e77d754a912078af8e47b36c9  -
$ 

It can be better to check only the relevant partition(s),

$ lsblk -f /dev/sdc
NAME   FSTYPE      LABEL UUID                                 MOUNTPOINT
sdc                                                           
├─sdc1                                                        
├─sdc2 vfat              34D9-D113                            
├─sdc3 ext4              3c66d05d-bc02-4a1e-baca-e227a161e345 
└─sdc4 crypto_LUKS       371f0cbc-3f6f-49dd-9fc4-4cdf91cb15c9 

in this case partition #4,

$ sudo pv /dev/sdc4 | md5sum
1,66GiB 0:00:46 [36,2MiB/s] [================================================>] 100%            
35d33ae006c90b47b2e7b9aacb7f9bd7  -

Please remember to unmount the partitions on the USB stick before checking the md5sum.

Cloning a drive with a safety belt

You can clone from one USB stick to another stick (or card or SSD, any mass storage device) with mkusb-dus which also uses pv if installed and watch-flush to monitor the progress of the cloning operation. Assuming the source drive is sdx, run

dus /dev/sdx

It will help you identify the correct target drive (help you avoid overwriting the wrong drive), so you do not specify the target on the command line.

cm flag
Appreciate the help on this one. Can `pv` be used for anything else or is it purely for creating a hash? As this would be a nice progress bar to have to show the progress of cloning the stick.
sudodus avatar
jp flag
You can pipe via `pv` so yes, you can use it for 'anything'. But when writing to slow media, e.g. USB pendrives, there will be buffering in RAM, and you should also run `sync` and wait while flushing the buffers. In the [mkusb](https://help.ubuntu.com/community/mkusb) package there is a shellscript `watch-flush`, that you can use separately if you wish. -- And you can use `mkusb-dus` to clone the stick and see progress (built-into mkusb).
sudodus avatar
jp flag
Please be aware of other cloning tools and the special feature 'backup partition table' of GUID partition tables. Read more about it at [this link](https://askubuntu.com/questions/958242/fastest-way-to-copy-hdd/958248#958248).
cm flag
Tried to get Clonezilla to work and for some reason didn't quite work.... May revisit it but I'm already so investing in building my own script I may just continue with what I have :D. Thanks for your help
sudodus avatar
jp flag
@cameron.g, You are welcome and good luck with your own script :-) /You can borrow things from my mkusb if you wish; it is mainly bash./
sudodus avatar
jp flag
@bac0n, Have you found a bug in mkusb? I'm sorry, but I don't understand. Please explain the context.
sudodus avatar
jp flag
@bac0n, Thanks for the heads up. I'll check for it, but would be happy, if you tell me what you found :-)
bac0n avatar
cn flag
@sudodus In the likelihood there is a file named, e.g., `a$` (slim to zero, but still) `grep [a-z]$` will expand to `grep a$` (globbing), which probably not desirable. The pattern should be quoted to avoid glob expansion.
sudodus avatar
jp flag
@bac0n, I have [tried to] quote all the grep 'patterns' in the current versions of mkusb: mkusb-dus and mkusb-plug. See https://github.com/sudodus/tarballs or if you use mkusb: upgrade via `ppa:mkusb/unstable` to version 12.7.2. If/when you have the time, please check if you find some problematic command that I have not yet identified and fixed.
bac0n avatar
cn flag
sure ..........
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.