Join Log in
Score:1
Controvi avatar Ubuntu

rsync error "rejecting excluded file-list"

sn flag
Controvi

So recently we updated 2 production servers and 1 development server for our websites. All are running Ubuntu 20.04. We also updated our build server that contains an installation of Jenkins. This one runs Ubuntu 18.04. The build server is only used internally and is not accessible from outside our own network. In our build process we use RSYNC to copy over all the files to the correct place on the production/development server.

To clarify, I am no Linux expert so the things I state below and the troubleshooting I have done are all from google searches and a limited knowledge of how Linux works.

This is the rsync command we use:

rsync -rltDv  --chmod=ug=rwX,o=rX -e ssh --exclude-from='exclude-list.txt' --delete --filter='protect wordpress/.htaccess' --filter='merge protect-list.txt' ./ jenkins@${WEB_SRV_ADDRESS}:/var/www/test

And this is the error we are getting:

ERROR: rejecting excluded file-list name: wp-content/plugins/[plugin name]
       /[subfolder]/[subfolder]/[subfolder]/.gitkeep
rsync error: protocol incompatibility (code 2) at flist.c(912) [receiver=3.1.3]

The versions of rsync in use:

  • prod & dev server (Ubuntu 20.04) > 3.1.3 protocol version 31 (latest)
  • build server (Ubuntu 18.04) > 3.1.2 protocol version 31 (latest)

What I have tried so far and its results
(after every test I returned everything to its original state)

  • Downgraded the build server version of rsync to 3.1.1
    • Still error
  • Downgraded the prod & dev version of rsync to 3.1.2
    • Still error
  • Returned Build server to backup before update
    • Still error
  • Returned dev server to backup before update
    • Solved error. But weirdly enough the versions of rsync were the same before and after the update. Which made me think and I verified that in the updated packages, rsync was not one of the things being updated
  • Changed the .gitkeep file mentioned in the error message to a php file
    • new error message
    • ERROR: rejecting excluded file-list name: wp-content/plugins/[plugin name]/
[subfolder]/[subfolder]/[subfolder]/test.php
  • Tried removing the .gitkeep file mention in the error message
    • Moves the error to the next folder that contains a .gitkeep file

The protocol error, as far as I have found online, is giving because rsync echoed an error message to the terminal and thus the terminal is not "clean".

I'm completely out of ideas after 2 days of searching. Hope someone here recognizes the issue or has ideas for me.

[**EDIT - contents of exclude-list.txt ** ]

cache/*
/.git
/.gitkeep
/.gitignore
/.gitmodules
/node_modules
/composer.json
/composer.lock
/Jenkinsfile
/gulpfile.js
/package.json
/package-lock.json
/protect-list.txt
/vendor
/get-composer.sh
/exclude-list.txt
wordpress@tmp
api-v2@tmp
/vite.config.js

[**Update- Debug info added ** ]

[receiver] hiding directory folder1/folder2/folder3/folder4/folder5/folder6 because of pattern folder1/folder2/folder3/folder4/folder5/folder6/  
[receiver] send_msg(3, 107)  
[generator] send_msg(3, 107)  
ERROR: rejecting excluded file-list name: folder1/folder2/folder3/folder4/folder5/folder6
353
1 + 6
ssh
uz flag
Jos
What exactly is in the file `exclude-list.txt`? It sounds as if there is (somehow) a syntax error in there.
0
Reply
Controvi avatar
sn flag
Controvi
I added the content of the exclude-list.txt to the question.
0
Reply
meuh avatar
cn flag
meuh
Try adding `--debug=ALL4` and looking for a clue in the log lines before the error.
0
Reply
Controvi avatar
sn flag
Controvi
Sadly doesn't give much more info. Updated the question with debug info
0
Reply
meuh avatar
cn flag
meuh
I tried understanding what provokes the message in the source, but it is too complex for a simple perusal. I get the feeling that it may be that because of `--delete`, the sender wants a file or dir deleted, but there is a `protect` or `exclude` entry on the receiver for that name. Try looking for differences in `protect-list.txt` entries (and the matched files or dirs) or the exclude list at both ends of the connection. Read `man rsync` section *PER-DIRECTORY RULES AND DELETE*.
0
Reply
Controvi avatar
sn flag
Controvi
Well after a lot o of searching I fixed it. though i still don't really know why it started in the first place, i changed some stuff and the debug flag you send me helped big time for that/ thanks. added the answer to the question below
0
Reply
Score:1
Controvi avatar Ubuntu
sn flag
Controvi

Ok, so after a bunch of searching the web and using the handy --debug=ALL4 (thanks @Meuh) I found out that the issue was not with the excludes but with the protected. It took some trial and error but eventually got it working.

I still don't really know what caused the issue all of a sudden but I am guessing that the mess of a command I have for RSYNC was a bad line to start with. It seems that updating Ubuntu caused the system to handle the RSYNC request a little differently or at least made something that was bad but working break.

Reading online that "having folders excluded would mean they will not be send to the receiver and it will delete files and folders that are not in the received file folder list" kept me adamant that I needed a protected list. Seems that is NOT the case as folders that are excluded WILL be ignored on the receiver end. I did some test runs with a different folder, very basic, and everything started working as expected.

I fixed our issue by changing this line command:

rsync -rltDv  --chmod=ug=rwX,o=rX -e ssh --exclude-from='exclude-list.txt' --delete --filter='protect wordpress/.htaccess' --filter='merge protect-list.txt' ./ jenkins@${WEB_SRV_ADDRESS}:/var/www/test

To this:

rsync -av --chmod=Du=rwx,Dg=rwx,Do=rx,Fu=rwx,Fg=rwx,Fo=rx,o=rX --filter='merge .rsync-filter' --delete ./ jenkins@${WEB_SRV_ADDRESS}:/var/www/test/

Where now I created a .rsync_filter file that looks like this:

- cache
- /.git
- /.gitkeep
- /.gitignore
- /.gitmodules
- /node_modules
- /composer.json
- /composer.lock
- /Jenkinsfile
....
- wp-content/media
- wp-content/uploads
....

The final 2 lines and a bunch more were protected at first but now excluded. Things work perfectly now and everything is as expected

+ 0
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.