Score:2

How to batch delete the first n bytes of a series of .jpg files to fix corrupt headers

fm flag

Situation: I have a folder filled with .jpg files that have corrupt headers: .jpg files need to start with FF D8 FF E0 to be interpreted as images, but my files have random values in the first four bytes followed by FF D8 FF E0. I have been manually deleting the first four bytes with a browser based hex editor: https://hexed.it/. It works nicely, but I can only fix one file at a time, which is tedious.

QUESTION: Is there a way to use the command line to batch edit these files? What command would I use? I need specific instructions, because I lack CLI experience.

Here is an example of my situation: I have 50 jpg files in a folder called KIimages. The files are all called "KIimage001", "KIimage002" and so on. I know how to open a folder in the terminal and list its contents. Then I get stuck. What do I do next to automate opening each image, deleting the first four bytes, and re-saving each file?

What I have tried: I read several articles/posts about writing scripts or using commands to do similar tasks, but but no instructions were clear or specific enough to be of any help, or the suggestions posted were above my skill level, which is basic.

I tried using the file command, and the head command, I also read about using something called dd, but the solutions were both too complicated, and not specific enough to my situation. Is there any way to do with simple terminal commands what I have described above? Thanks. FYI I am using Pop!_OS, in case that is relevant.

Score:0
ca flag
for file in /foo/*
    do
      if [ -f $file ]
      then
        tail -c +5 $file > $file.truncated && mv $file.truncated $file
      fi
    done

In response to the comment, here is the link to the video, explaining the solution and how to use it in every detail: https://youtu.be/Ta5ahcieegg

Here is short step by step instruction:

  1. Create a file in your favorite text editor - nano, vim, etc.
  2. In terminal you can use command touch fix.sh for example.
  3. Copy-paste code above into this file.
  4. Make sure you update the path to your files in the first line!
  5. Save the file.
  6. Make it executable by entering chmod +x fix.sh in terminal.
  7. Execute the script in terminal by this command: ./fix.sh

For detailed demo, please, see video from the link above.

Hannu avatar
ca flag
Suggested addition: `if [ "$(od -t x1 -w4 $file | head -n 1)" = "0000000 FF D8 FF E0" ] ; then echo "OK"; else echo "Not OK, do the thing!"; fi`
Chuctanunda Spiderbone avatar
fm flag
Thank you, Denis and Hannu for your suggestions. Unfortunately, I do not have the expertise to know what to do with these strings of code. I spent quite a lot of time googling them to find out what they actually do, and how to employ them, but I finally realized it was much quicker to manually delete the required bytes using hexed.it. Unfortunately, just providing code strings with no context or explanation is not helpful for many people who are either new to Linux, or the command line. Perhaps you could add information on what this code actually does, steps to take to employ it. Thanks.
Denis Rasulev avatar
ca flag
Hey, @ChuctanundaSpiderbone! This was a bit unexpected :) Anyway, I've updated the answer and created a small video explaining the whole process. Enjoy!
Chuctanunda Spiderbone avatar
fm flag
Thank you, @DenisRasulev. This is exactly what I needed. I will study your updated answer and report back with my experience. Thank you so much for making a video. It will be very helpful.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.