Join Log in
Score:3
Phil H avatar Ubuntu

How do I compress a folder while preserving file ownership

mx flag
Phil H

I have a docker server I want to backup. My deployment folder has very specific ownership and permissions to support my containers:

$ ls -lhaF /opt/docker
total 32K
drwxr-xr-x  7 devops devops 4.0K Aug 23 02:34 ./
drwxr-xr-x  6 root   root   4.0K Aug 23 04:20 ../
drwxrwxr-x  2 devops devops 4.0K Aug 21 00:00 .certs/
drwxrwxr-x  2 devops devops 4.0K Aug 23 03:53 .scripts/
-rw-rw-r--  1 devops devops 1.2K Aug 21 01:52 docker-compose.yml
drwxrwxr-x  4   1000   1000 4.0K Aug 21 02:05 minecraft/
drwxrwxr-x  4 devops devops 4.0K Aug 20 23:38 pihole/
drwx------ 19     70     70 4.0K Aug 19 01:31 postgres/

I then try to compress this folder using the following command. From what I've read preserving file permissions/ownership requires running tar as root/sudo:

$ sudo tar -czpf "/tmp/server-backup.tar.gz" --directory="/opt/docker" .

This command compresses the folder but inspecting the .tar.gz it shows that ownership of all folders are now root:

$ tar -tvf /tmp/server-backup.tar.gz
drwxr-xr-x root/root         0 2021-08-23 12:10 ./
drwxr-xr-x root/root         0 2021-08-23 12:10 ./minecraft/
-rw-r--r-- root/root         2 2021-08-23 12:10 ./minecraft/whitelist.json
-rw-r--r-- root/root       111 2021-08-23 12:10 ./minecraft/usercache.json
-rw-r--r-- root/root      1204 2021-08-23 12:10 ./minecraft/server.properties
-rw-r--r-- root/root  43626592 2021-08-23 12:10 ./minecraft/minecraft_server.1.17.1.jar
-rw-r--r-- root/root        68 2021-08-23 12:10 ./minecraft/eula.txt
-rw-r--r-- root/root         2 2021-08-23 12:10 ./minecraft/banned-players.json
-rw-r--r-- root/root         2 2021-08-23 12:10 ./minecraft/banned-ips.json
# -- Other minecraft files
drwxr-xr-x root/root         0 2021-08-23 12:10 ./.certs/
# -- Certs folder contents redacted
drwx------ root/root         0 2021-08-23 12:10 ./postgres/
drwx------ root/root         0 2021-08-23 12:10 ./postgres/pg_subtrans/
-rw------- root/root      8192 2021-08-23 12:10 ./postgres/pg_subtrans/0000
drwx------ root/root         0 2021-08-23 12:10 ./postgres/pg_multixact/
drwx------ root/root         0 2021-08-23 12:10 ./postgres/pg_multixact/offsets/
-rw------- root/root      8192 2021-08-23 12:10 ./postgres/pg_multixact/offsets/0000
drwx------ root/root         0 2021-08-23 12:10 ./postgres/pg_multixact/members/
-rw------- root/root      8192 2021-08-23 12:10 ./postgres/pg_multixact/members/0000
drwx------ root/root         0 2021-08-23 12:10 ./postgres/pg_xact/
-rw------- root/root      8192 2021-08-23 12:10 ./postgres/pg_xact/0000
# -- Other postgres files
drwxr-xr-x root/root         0 2021-08-23 12:10 ./pihole/
drwxr-xr-x root/root         0 2021-08-23 12:10 ./pihole/etc-pihole/
-rw-r--r-- root/root         0 2021-08-23 12:10 ./pihole/etc-pihole/custom.list
-rw-r--r-- root/root   5201920 2021-08-23 12:10 ./pihole/etc-pihole/gravity.db
-rw-r--r-- root/root       485 2021-08-23 12:10 ./pihole/etc-pihole/setupVars.conf
-rw-r--r-- root/root         0 2021-08-23 12:10 ./pihole/etc-pihole/setupVars.conf.update.bak
-rw-r--r-- root/root   1812161 2021-08-23 12:10 ./pihole/etc-pihole/list.1.raw.githubusercontent.com.domains
-rw-r--r-- root/root     73728 2021-08-23 12:10 ./pihole/etc-pihole/pihole-FTL.db
-rw-r--r-- root/root         0 2021-08-23 12:10 ./pihole/etc-pihole/pihole-FTL.conf
-rw-r--r-- root/root        37 2021-08-23 12:10 ./pihole/etc-pihole/local.list
-rw-r--r-- root/root        95 2021-08-23 12:10 ./pihole/etc-pihole/list.1.raw.githubusercontent.com.domains.sha1
-rw-r--r-- root/root        20 2021-08-23 12:10 ./pihole/etc-pihole/localbranches
drwxr-xr-x root/root         0 2021-08-23 12:10 ./pihole/etc-pihole/migration_backup/
-rw-r--r-- root/root        65 2021-08-23 12:10 ./pihole/etc-pihole/migration_backup/adlists.list
-rw-r--r-- root/root       618 2021-08-23 12:10 ./pihole/etc-pihole/dns-servers.conf
-rw-r--r-- root/root        20 2021-08-23 12:10 ./pihole/etc-pihole/GitHubVersions
-rw-r--r-- root/root        44 2021-08-23 12:10 ./pihole/etc-pihole/localversions
drwxr-xr-x root/root         0 2021-08-23 12:10 ./pihole/etc-dnsmasq.d/
-rw-r--r-- root/root      1475 2021-08-23 12:10 ./pihole/etc-dnsmasq.d/01-pihole.conf
drwxr-xr-x root/root         0 2021-08-23 12:10 ./.scripts/
-rwxr-xr-x root/root      1638 2021-08-23 12:10 ./.scripts/create-backup.sh
-rwxr-xr-x root/root       511 2021-08-23 12:10 ./.scripts/new-cert-pihole.sh
-rwxr-xr-x root/root       345 2021-08-23 12:10 ./.scripts/fix-permissions.sh
-rw-r--r-- root/root      1170 2021-08-23 12:10 ./docker-compose.yml

If I try to extract the .tar.gz file we can confirm that all ownership has been lost:

$ sudo mkdir /tmp/server-backup
$ sudo tar -xzpf /tmp/server-backup.tar.gz --directory=/tmp/server-backup
$ ls -lhaF /tmp/server-backup
total 32K
drwxr-xr-x  7 root root 4.0K Aug 23 12:10 ./
drwxrwxrwt 13 root root 4.0K Aug 23 12:16 ../
drwxr-xr-x  2 root root 4.0K Aug 23 12:10 .certs/
drwxr-xr-x  2 root root 4.0K Aug 23 12:10 .scripts/
-rw-r--r--  1 root root 1.2K Aug 23 12:10 docker-compose.yml
drwxr-xr-x  4 root root 4.0K Aug 23 12:10 minecraft/
drwxr-xr-x  4 root root 4.0K Aug 23 12:10 pihole/
drwx------ 19 root root 4.0K Aug 23 12:10 postgres/

From what I've found, preserving ownership and permissions is supposed to be default behavior for tar. Is there something I'm missing here? I'd like to be able to backup the state of my server without messing up the delicate file ownership required by some of the docker containers I run.

System information:

$ lsb_release -d
Description:    Ubuntu 20.04.3 LTS
$ uname -a
Linux rpi-1 5.4.0-1042-raspi #46-Ubuntu SMP PREEMPT Fri Jul 30 00:35:40 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
84
1 + 6
tar
muru avatar
us flag
muru
What happens if you don't use the `-p` flag? Just `czf` and `xzf`?
1
Reply
Artur Meinild avatar
vn flag
Artur Meinild
I just tested in a random folder, and if I run `tar` on 2 files with the `-czpf` flags, it correctly preserves permissions.
3
Reply
cn flag
Rinzwind
Can confirm: that is what the `-p` (small letter not the capital P) is for. there is also `--preserve-permissions` / `--same-permissions` and `--atime-preserve` and `--same-owner` "From what I've read preserving file permissions/ownership requires running tar as root/sudo:" nope you have options to prevent this.
2
Reply
sudodus avatar
jp flag
sudodus
I notice that you specify Ubuntu 20.04.3 LTS for Raspberry Pi. Point release 3 is not yet released (but very close to be released). Which version of tar are you running? Check with `apt-cache policy tar`; maybe you have found a bug, because your results are contrary to what many of us have experienced. It might be a new bug, but it is also possible that the bug has affected the RPi version of tar for years, because most of us use the PC version 'amd64' and would never be affected by it.
1
Reply
Artur Meinild avatar
vn flag
Artur Meinild
I just tested again on Raspberry Pi, and permissions are still preserved for me. Running Ubuntu `20.04.3 LTS` (have been pushed for existing installations) and tar `1.30+dfsg-7ubuntu0.20.04.1` on both platforms.
1
Reply
Phil H avatar
mx flag
Phil H
After further investigation, it looks like the culprit isn't actually the `tar` command. I was using `cp` to copy my deployment folder before compressing to avoid read errors. I didn't realize that you must explicitly preserve permissions with something like `cp -Rp`. Thanks for helping me narrow this down.
2
Reply
Score:1
Phil H avatar Ubuntu
mx flag
Phil H

Apparently, tar isn't the culprit here. This was part of a larger script to backup my deployment folder. I was copying the folder before compressing to avoid read inconsistencies while the containers were still running. The script contained the following:

cp -R "/opt/docker" "/tmp/server-backup"
tar -czf "/tmp/server-backup.tar.gz" --directory="/tmp/server-backup" .

File ownership was being lost during the folder copy. The -p flag is required to preserve permissions/ownership on a folder copy. The correct command would have been this:

cp -Rp "/opt/docker" "/tmp/server-backup"
tar -czf "/tmp/server-backup.tar.gz" --directory="/tmp/server-backup" .
0 + 3
sudodus avatar
jp flag
sudodus
Thanks for showing and explaining the solution :-)
0
Reply
Artur Meinild avatar
vn flag
Artur Meinild
I usually copy with the `-a` (archive) option, which equals to `-dR --preserve=all`. This is one single option that preserves everything you want to preserve for a backup - I even have an alias for it: `alias cpa='cp -a'`
1
Reply
Phil H avatar
mx flag
Phil H
@ArturMeinild Thanks! I'll keep that in mind.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.