Join Log in
Score:0
António Casqueiro avatar Server

md5sum is the same but gunzip test fails

cn flag
António Casqueiro

I've automated a process that involves fetching a file from a backup server and them processing it.

Sometimes the processing phase fails and I thought it might be due to the source file being corrupted.

I've found that even when it fails, the md5 of copied file matches the md5 of the file in the backup server. So as an additional sanity check I decided to test the integrity of the .gz file.

Storage machine

md5sum file-2021-08-31-05_15.sql.gz 
962b4916651daf54372f25d55e7ee2d2  file-2021-08-31-05_15.sql.gz

gunzip -v -t file-2021-08-31-05_15.sql.gz 
file-2021-08-31-05_15.sql.gz:    OK

Machine 1

md5sum file-2021-08-31-05_15.sql.gz 
962b4916651daf54372f25d55e7ee2d2  file-2021-08-31-05_15.sql.gz

gunzip -v -t file-2021-08-31-05_15.sql.gz 
file-2021-08-31-05_15.sql.gz:   
gzip: file-2021-08-31-05_15.sql.gz: invalid compressed data--crc error
gzip: file-2021-08-31-05_15.sql.gz: invalid compressed data--length error

Machine 2

md5sum file-2021-08-31-05_15.sql.gz 
962b4916651daf54372f25d55e7ee2d2  file-2021-08-31-05_15.sql.gz
gunzip -v -t file-2021-08-31-05_15.sql.gz 
file-2021-08-31-05_15.sql.gz:    OK

Question: How is it possible the "file-2021-08-31-05_15.sql.gz" has the same md5 in the 3 machines, but in only one of them the "gunzip -v -t" fails?

51
0 + 1
scp
md5
Nikita Kipriyanov avatar
za flag
Nikita Kipriyanov
Obviously, something wrong with CPU or memory on that faulty machine. Why MD5 calculation is correct then? Probably it is not using that buggy memory area or processor instruction, just by occasion, but gzip does and fails. This is why memory and CPU testing tools run a fair amount of tests and a failure in just one is enough to declare a part buggy.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.