Questions tagged as ['rsync']

Good day, friends!

I've got a locally configured ~/ssh/config file with all the hosts set up as they should. I am using them successfully when doing my ssh from the local computer like ssh dev. But when I am on a remote server I cannot do it and get this error Could not resolve hostname XYZ: Name or service not known. I have configured my local keys to be passed to remote system with ForwardAgent yes

I got two folders, a source S with files (and subfolders and so on):

• a.txt
• b.txt
• c.txt
• ...

and a destination folder D with files:

• a.txt
• b.txt
• b1.txt
• c.txt
• c2.txt
• ...

and I want to delete all files in D that are already present in S, so that D contains just files that are not present in S.

Any ideas, how I can achieve this with standard tools (rsync, robocopy, ...)?

P.S.: In real world S is abo ...

I am using a script to make a backup of a running Debian Stretch based Linux Distro (NextCloudPi)

The method in the script that does the backup uses rsync.

backup()
{
  mntimg
  sync
  rsync -aDH --partial --numeric-ids --delete --force --exclude "${MNTPATH}" --exclude '/dev' --exclude '/lost+found' --exclude '/media' --exclude '/mnt' \
--exclude '/proc' --exclude '/run' --exclude '/sys' --exclude '/t ...

I need to rsync a file from remote hosts to my local machine over multiple jumphosts. I'd like to use shared SSH connections so that I don't have to submit 2FA multiple times at some jumphosts.

The idea is I create a master connection

ssh user@jumphost01

then other connections

ssh -J user@jumphost01,user@jumphost02 root@target01
...

Shared connections control files in ~/.ssh/ look as follows

co ...

I have hyperbackup running from source NAS to destination NAS with the below settings

1. Backup type - Rsync
2. Options - reserve backup file at destination
3. With this i have a backup method that results in a file/fodler structure in the 2nd NAS, withut the hyperbackup proprietary format or tools

I have the below cronjob on source NAS via task schedule which deletes files on the destination NAS which are  ...

rsync: [generator] failed to set permissions : Operation not supported (95)

I'm running ubuntu 21.10 and facing the same problem as the above link, but could not follow the solution steps, for e.g. installing libssl-dev gives me this error below:

[~/tmp/rsync-be3d6c0fbbd07781bbae6261cda109f8f08c031b]# apt install libssl-dev
Reading package lists... Done
Building dependency tree... Done
Reading state  ...

basically the title. I'm lacking unix basics but on windows if I backup with robocopy I wouldn't want to preserve ownership bc then I would have to take ownership of it again anyway when replaying the backup. (default is actually data, attr, timestamps)

I'm trying to backup a macbook in order to reset it and I'm hesitant to use the global archive flag -a. Will I have to take ownership of the file ...

I updated my file server VM (VMware) from CentOS 7 to RHEL 8. I doubled the CPU's and memory on the new system, but for some reason rsync's seem to bog the system down considerably. To the point where users have trouble connecting over ssh. Total CPU usage is low (never above 1%) iotop shows nothing from the user doing the rsync, iftop show some TX/RX, but it's generally around 12Mb TX, 4Mb RX for a 1Gb ...

CONTEXT : rsync with very large number of files to sync

WHAT HAPPENS : rsync encounters errors, but just continues.....

WHAT I DO : rsync -Pav path1 path2

ISSUE : I'm booted in a recovery mode and the recovery drive have no space left...

QUESTIONS : is there an option to make rsync stop at first error so I can see what's happening ?

I TRIED to look man page of rsync, but cannot find any option that would  ...

How do we find where the rsync is scheduled? I have a rsync that's done in server A to server B, and I need to find in server A where rsync is called and where and how it's scheduled. Is there a file for this where the config are set like for cronjobs? I have some performance issues in server B and I need to reschedule the rsync so that it only happens outside of peak hours.

I have an aws ec2 instance and I am trying to setup github action. My action is not able to sync. The error I get is

Warning: Permanently added 'XXXXXXXXXX.compute.amazonaws.com,XX.X.XX.XX' (ECDSA) to the list of known hosts.
sending incremental file list
rsync: failed to set times on "/var/www/html/wp-content/themes/XXX-theme/.": Operation not permitted (1)
./
rsync: failed to set times on "/var/w ...

I have an issue where 2 processes take 99% of the CPU, but that leaves me the mysqld process that takes 600% of the CPU and a set of php-fpm processes that takes no more than 25% of the CPU according to top (2 process alive and 603 processes sleeping, one of them is the mysqld and the other seems to be a php-fpm, so it's not coming from too many clients), so I am wondering what might cause the high CPU  ...

Is there a way to find out what files are being moved by rsync and when and where it's scheduled? I was told by the senior administrator that there was a rsync process that allowed us to back the data from server A to server B. Is there a way to see from server B where the rsync process is called and how it's scheduled? I know it's scheduled on server A, but I would like to know if from server B, I can  ...

I am using rsync like below to copy the whole filesystem of the remote server with the hostname "beton" to a local directory. I would like to exclude virtual directories like proc, but even if I use the exclude parameter, it is still copied!

sudo rsync -aAXv root@beton:/ --exclude={/dev*,/lost+found*,/media*,/proc*,/run*,/sys*,/tmp*}  /mnt/big/beton_backup

I also tried other possibilities, but the exc ...

I'm using rsync for incremental backups, using the --link-dest option to point to the previous backup so that unchanged files are hard linked to there.

This is working, but not for all files. For example, I've a directory in the backup containing files that haven't been modified in over three years. But for some bizarre reason, only some of them are hard links.

Inconveniently, the larger files are all ...

I want to use rsync to backup data from webserver to Truenas. The webserver runs Debian 11 and Truenas is at 12.0-U5 The backup is saved on the webserver in an encrypted format using restic. Since it's already encrypted I want to use rsync directly to avoid ssh overhead.

On Debian rsync is at version 3.2.3 and on Truenas 3.1.3 now when I use

rsync -rdt rsync://rsync@truenas

I occasionally get a connectio ...

I have a backup script on my WSL Ubuntu that I use manually to backup my linux systems to a USB drive (NTFS volume mounted on WSL).

I'm running into an issue with --link-dest that I haven't had before in other backup scripts; but it's been a while since I've written anything in bash.

rsync --progress -rlDvuRH --modify-window=1 link-dest=$LAST/ -e "ssh -i /home/username/.ssh/id_rsa" --exclude='*home/.e ...

By mistake we did a

chown www-data:www-data / -R

on a production server.

We do have a backup, but as only the permissions are wrong we want to restore only the permissions from the backup, not the file content.

Is there a simple command for this, for example with rsync?

I am making a cron job on my server node that uses haproxy base concated cert file from letsencrypt. however when I am at home I have same dns pointing to the home server which is actually hosting the website so the domain I access is directly via local server IP me being on same shared network. and since the certbot runs on proxy node server on digitalocean, I need to rsync the uncocatenated cert files ...

I have an ubuntu server (backup server) in a remote location. It has some bash scripts that create backups. The server's been working for years. Recently, it started "hanging", meaning that I can't login via ssh anymore. Also it doesn't reply to pings.

Rebooting the server helps, it works again. I formatted the internal drive where backups are stored and launched a simple rsync operation (Via ssh ...

I know that i can add in Rsync's config file

read only = no

But when I use the synchronization command on the client, rsync tells me

ERROR: module is read only

If I change the value of read only to “no”, the client will automatically upload files when synchronizing files with the server

So what should I do to allow the client to only download changed files from the server and not allow local files  ...

I use rsync to backup the qcow2 disks of my virtual machines. However, I would like to be able to get an incremental backup so that I don't have to copy the entire disk every time. for now I use rsync -hav --progress / source / target, the execution time is always the same even if the second time the disk has already been copied and it should take much less

UPDATE

doing some additional research I di ...

I have an issue with files attributes since I changed my company's file system from an old ubuntu 12 to a Synology NAS.

I initially mounted NFS shares on my old file system, then with cp I copied everything, everything was ok But i forgot to preserve the attributes, and we lost especially the date, the dates became the copy date, and not the real date.

I found the rsync command rsync -vrt --size-onl ...

I want Host B to set up a ssh tunnel to Host A so it can rsync with Host C. Only Host B has keys. All three machines use the same key.

I've tried this:

ssh -i ~/.ssh/key -A -R localhost:50000:Host_A:22 Host_C 'sudo -E -s rsync -a -e "ssh -v -l admin -p 50000 -o StrictHostKeyChecking=no" --rsync-path="sudo rsync" /path/ localhost:/path'

However, I'm getting an ssh permission denied error when runni ...

0

I wish to synchronize letsencrypt credentials from host S to Host D using an ansible task running on host H.

My current task looks like this:

- name: Synchronize local letsencrypt directory
  ansible.posix.synchronize:
    src: /etc/letsencrypt
    dest: /etc/letsencrypt
    archive: true
    checksum: true
    delete: true
    recursive: true
    become_user: myuserid
    rsync_path: "sudo rsync"
  be ...

all of a sudden I can't login to my server from my Mac Terminal. On my Linux machine normal ssh login still works, except rsync over ssh also gets stuck.

This is what happenes when I try to connect from my Mac:

ssh -v user@xx.xx.xx.xx
OpenSSH_7.8p1, LibreSSL 2.6.2
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug1: Connecting to xx ...

After starting a backup it goes for some time before failing, which is what I'm finding hard to understand. I created a fresh test backup that worked flawlessly, it was just a couple of files, but for the whole drive it fails after a while, in this case it received nearly 42 gigabytes before failing

Here's the error:

rsync_bpc: connection unexpectedly closed (41767840413 bytes received so far) [receiv ...

I want to run an rsync job that copies data from A to B. The data at the destination will be deleted after processing. However, the processed data should no longer be copied. Is there a way that rsync remembers the deleted data on the target and only copies the new data in the source? Maybe there is a way to make a ongoing list and exclude the data from the list?

I want to use rsync to backup databases from my webserver.

I created a cronjob on the webserver that creates a mysqldump of the database and names it with the current date in the beginning every night. Files older than 30 days will be deleted.

Now I want to backup only the databasefile of the current date to my NAS via a cronjob.

How can I only include that one file? Or vise versa, how can I excl ...

First I thought that rsync isn't syncing everything described here Limitations of rsync? Not syncing everything

Now after a day I found out that it was actually still working/ running and now all the files are there. The problem is: My zip is instead of 1gb only 200mb.

Because the zip was created I thought rsync is "finished" and just doesn't download everything. My logic was: tar only gets executed w ...