Score:0

Why is rsync only hard linking some unchanged files?

id flag

I'm using rsync for incremental backups, using the --link-dest option to point to the previous backup so that unchanged files are hard linked to there.

This is working, but not for all files. For example, I've a directory in the backup containing files that haven't been modified in over three years. But for some bizarre reason, only some of them are hard links.

Inconveniently, the larger files are all copies (i.e. there's only one link to the file, checked via ls -l). But that's also the case for some smaller files, and indeed some hard linked files are larger than some copied files.

There doesn't seem to be any pattern that allows me to predict which will be copies and which hard links. Filename length and file size seem irrelevant, as does modification date: there's a mix of all of these in both the copied and hard linked lists. That said, the files do seem to be consistent across multiple backups, so what happened for one backup seems to have happened for the next backup too.

Is there some attribute (technical or otherwise) of a file, some feature, that would cause rsync to copy it rather than hard link it?

EDIT 1: Mentioning "attribute" made me wonder whether there was some attribute that ls -l doesn't list that might have an impact, but doing lsattr in the source directory shows that all files have identical attributes.

EDIT 2: I said previously (now deleted) that the permissions were all the same, but that was wrong. The permissions were the same in the destination (backed up) directory. I am using --perms (to preserve permissions), so I don't know why permissions are not being preserved. I was also previously running as a non-root user, but am now trying as root in case that makes a difference, but the permissions still are not being preserved, which could perhaps be why some files look like they have changed - the file hasn't but its permissions apparently have (at least, when compared to the previous backup with its incorrect permissions).

EDIT 3: I now think it's something to do with my CIFS server. There's something in the mount.cifs man page about the file_mode option: "If the server does not support the CIFS Unix extensions this overrides the default file mode." When I run mount command with no arguments to list the mounts, the options listed include file_mode=0755 and dir_mode=0755 which ties in with what I see. I can't chmod a file on the mount, so those files that didn't originally have 0755 permissions are deemed to have changed and so are copied again - and once again get the wrong permissions in the backup because of the mount.

Score:0
id flag

More of a workaround than a solution, I now avoid using --perms/-p or anything that implies it. Obviously then my permissions are not copied across, but at least it doesn't think an unchanged file is changed just because its permissions are different.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.