Join Log in
Score:0
jona avatar Server

Limitations of rsync? Not syncing everything

id flag
jona

I'm pretty new to all of this, so this could maybe be an easy fix.

To break it down:
I set up a cronjob on our NAS which syncs our webserver to a specific folder. There is a lot of stuff on the server with around 70.000 files and overall around 500mb.

On first run, all of the folders got created but only some (maybe the first two) contain something inside.
All the other folders are empty on the NAS (but on the webserver there are files and other folders inside).

Is there any limitation by default which I don't get? Time, filesize, logfilesize or file amount?

My script is the following:

USER="myusername"
SERVER="myserverip"
PORT="22"
SOURCE="shop"
TARGET="/volume1/BACKUPS/MyFolder/MySubfolder/data/"
LOG="/volume1/BACKUPS/MyFolder/MySubfolder/backup_data.log"

mkdir /volume1/BACKUPS/MyFolder/MySubfolder/

rsync -avz --progress -e "ssh -p $PORT" $USER@$SERVER:$SOURCE $TARGET --delete >> $LOG 2>&1
tar -zcvf /volume1/BACKUPS/MyFolder/archivename-$(date +%Y-%m-%d_%H_%M_%S).tar.gz -C /volume1/BACKUPS/MyFolder/MySubfolder/ .

The log is pretty insane with all the progress percentages and stuff. It's around 15mb (just the log). So this won't help much. But when I search the log for "error" I get no results.

121
0 + 4
jp flag
Dom
Do you have log in your system (like Out of memory) ? It should works.
0
Reply
in flag
Zoredache
I probably wouldn't use `--progress` if you are logging to a file. That is more of an interactive thing. I would probably also add a `set -e` to your script so if rsync fails, it won't try to do the tar.
0
Reply
jona avatar
id flag
jona
Thanks. The thing is: Some stuff gets downloaded and also zipped. If the tasks fails somewhere the nas will send me an error. It‘s a specific error where it says whats wrong and not just „task not completed“. But in this case I get no error at all, so it seems like rsync has no error (besides that fact that only 30% is downloaded)
0
Reply
9mjb avatar
ru flag
9mjb
I'd cleanup that log... so you can read it. I'd guess your issue is described there.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.