Score:1

xxd on Linux 22.04 LTS - Can I write one small file into a larger file at a given offset without copy and pasting contents of small file?

ai flag

I'm playing around with trying to set up patching copies of old GameBoy Pokemon games. I can do it all manually via other apps on Windows, and could maybe scrummage around for a Windows command line program to try this to speed up the process, but with Ubuntu having xxd already I figured I would play with this.

My end goal is to eventually have a bash script where it will read several source files to patch a copy of a base file.

But in learning xxd, I am struggling to find if that's possible. I can provide it $ echo "[offset]: [manually copy-pasted or typed in hex]" | xxd -r -c 256 "Source File.gb" but I am not sure if I can replace the manually copy-pasted stuff with the name of a "patch" file. Not to be confused with using the patch command, which I read in previous research doesn't work nicely for binary (hex) files, but more so for lined files.

I may just need to learn more about how the terminal works with whatever standard output or a buffer means. If anyone knows if that's possible in xxd, or can point me to a hex editor (linux/ubuntu only or multiplat) that has command line commands that would do what I envision, that would be great.

What I'd also like is the ability to have it patch more than 256 bytes at once. A reference file may accomplish that, but when done manually, you must have -c N where N is how many bytes you want to write. But -c is limited to 256. I had hoped that maybe -l instead of -c would let me write more, but it doesn't work, as shown in the screenshot here. The intended written content stops at the 30th byte and the rest of the file remains unchanged. The workaround of course is to break the steps into 256 byte chunks and just move the offset up by 0x100 for every broken chunk being applied. Picture of the command line and output of 4 commands. First is the failed use, second is the file results to verify previous command failed. Third is a correct use, fourth is the file results to verify previous command worked.

[i]Edit 3 or 4: Actually, looks like this can be fixed with using echo "..." | xxd -r -p - "Output.bin"[/i], the -p for "plain" seems to ignore any column restrictions and will take the stdinput in full. I was at least able to test write a new file that was over 1kb large.[/i]

To clarify what I have: I have known offsets that I want to modify in this file. xxd does that wonderfully, especially when I am only making short changes. I know exactly the contents I want to write at each offset, but it requires copy and pasting them into the terminal, including opening a .bin file via xxd -p "My Bin File.bin" and then copy-pasting that, removing any line breaks, into my writing command (echo stuff).

What I want to get to: The ideal is to be able to set up a bash script that reads "Change1.bin", "Change2.bin", "Change3.bin" and applies the changes at each offset, or at least prompts the user (me) for what hex values I want to write one at a time. So instead of having to type out echo "[offset]: [bytes]" | xxd -r -c 256 - "Base File.gb" it would just prompt "What hex do you want at offset 0x04000 ?". But much the same, that's me learning how to use standard output or some other reference value for the contents of the echo command, which I don't know yet.

Edit: Some brainstorming on trying to pass the file in:

I brainstormed for using xxd -p "MySmallPatch.Bin" | xxd -r -c256 -s 2C29D - "Pokemon Red Linux.gb" but it doesn't work, giving the error message "Sorry, cannot seek backwards". It might work if I had someway to do "0002C29D: " . xxd -p "MySmallPatch.Bin" | xxd -r -c256 - "Pokemon Red Linux.gb" to specify that offset as expected from the echo command, but I don't know how to set that prefix to the first part of the bash's output. Trying to do echo "0002C29D: " xxd -p "MySmallPatch.Bin" just gives the literal output, it doesn't run an xxd command, and thus pipes just that literal text if I try to pipe it.

Edit: I came across this post about the dd command being used to take an input file and overwrite a part of the output file, which is a step in the right direction for me. I don't recognize there should be any 256 byte limit like xdd appears to have.

https://unix.stackexchange.com/a/214824

Having discovered there's such a thing as unix.stackexchange, my question may be better suited there. No worries if this one goes unanswered, I recognize it's not distinctly Ubuntu specific.

Score:0
cn flag

Forget xxd. You've already found the answer.

So now we have the binary, we know where the string we want to change is, and we have a file with the replacement string in it.

Now we can use dd:

dd if=tmp of=test obs=1 seek=1460 conv=notrunc

This reads data from tmp (our "/tmp\0" file), writing it into our binary, using an output block size of 1 byte, skipping to the offset we found earlier before it writes anything, and explicitly not truncating the file when it's done.

tmp is the name of your raw binary patch.

test is the file you are directly patching (make a backup first).

The only thing not there is that if your offset to patch at is in hex, you write it as seek=$((0xoffset)).

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.