Discrepancy in file usage after rsync

NeDark

4/23/24, 1:17 PM

I copied files from an exFAT filesystem to an ext4 filesystem:

sudo rsync -avz /media/user/1TB-SD-EX/ -e ssh example.com:/data/pc/sd1tb

After successful run I check source and destination size with du -b

$ sudo du -ah -b -d0 /media/user/1TB-SD-EX/
800100388315    /media/user/1TB-SD-EX/

$ sudo du -ah -b -d0 /data/pc/sd1tb/
793592516059    /data/pc/sd1tb/

If I check number of items in with find and wc

$ sudo find /media/user/1TB-SD-EX/ | wc -l
126743

$ sudo find /data/pc/sd1tb/ | wc -l
126743

I know there is difference in real disk usage due to filesystem differences, but why is apparent usage different?

204

1 + 1

files

filesystems

rsync

disk-space-utilization

Romeo Ninov

4/23/24, 3:03 PM

To check the files use hash, `du` is not reliable.

Score:4

Server

anx

4/23/24, 3:21 PM

coreutils du - even when called with --apparent-size (implied by -b) - may add a non-zero size of directories to the total. This size will be 512 bytes for an empty FAT directory, but 4096 bytes for an empty ext4 directory, rarely matching for larger directories, thus a notable difference.

I would recommend to think twice before considering anything a proper backup that duplicates symlinks, hardlinks, acls, xattrs, device & special files to a different filesystem - possibly treating any of those differently. Otherwise, just trust that rsync knows what its doing. You do not validate a backup by asking if some other tool thinks the ones and zeroes match. You execute your documented & trained restore process, and then you validate some business logic on the restored result.

+ 2

NeDark

4/24/24, 2:17 PM

You say empty directories use more space in ext4, but size returned by `du -b` is actually bigger for extFAT

anx

4/25/24, 11:53 PM

@NeDark My mistake, I misread the question. Twice. I suppose the point that directories are one source of difference still stands, even though empty directories (requiring one block minimum) are not the best benchmark of the different space usage, as usually a folder structure does not consist of mostly empty empty directories.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Discrepancy in file usage after rsync

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.