Join Log in
Score:1
avatar Ubuntu

"Disk Full" After a rsync failure?

us flag
sn0ep

So I ran an rsync over night in a tmux sessions. It was to copy contents from one mount to another (USB disks).

Rsync command was:

rsync --info=progress2 /mnt/wdpassport /mnt/wdredpro

When I woke up this morning I saw that my / is completely full. While I'm actually sure it is not.

What can I do to solve this?

Output of: df at the moment of unmounted wdredpro 4tb

Filesystem                         1K-blocks      Used  Available Use% Mounted on
udev                                 8086632         0    8086632   0% /dev
tmpfs                                1626320      5056    1621264   1% /run
/dev/mapper/ubuntu--vg-ubuntu--lv  118882128 118746796          0 100% /
tmpfs                                8131596         0    8131596   0% /dev/shm
tmpfs                                   5120         0       5120   0% /run/lock
tmpfs                                8131596         0    8131596   0% /sys/fs/cgroup
/dev/nvme0n1p2                        999320    207296     723212  23% /boot
/dev/nvme0n1p1                        523248      8036     515212   2% /boot/efi
/dev/loop1                            101760    101760          0 100% /snap/core/11606
/dev/loop2                             56832     56832          0 100% /snap/core18/2074
/dev/loop0                            101888    101888          0 100% /snap/core/11420
/dev/loop3                             69248     69248          0 100% /snap/lxd/20326
/dev/loop4                             33152     33152          0 100% /snap/snapd/12704
/dev/loop6                            103424    103424          0 100% /snap/kotlin/61
/dev/loop5                             33152     33152          0 100% /snap/snapd/12883
/dev/loop7                             72064     72064          0 100% /snap/lxd/21029
/dev/loop8                             56832     56832          0 100% /snap/core18/2128
/dev/loop9                            102912    102912          0 100% /snap/kotlin/60
/dev/sdc2                         1953214336 435989632 1517224704  23% /mnt/wdpassport
tmpfs                                1626316         4    1626312   1% /run/user/1000

Output of: sudo fdisk -l

Disk /dev/loop0: 99.39 MiB, 104210432 bytes, 203536 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop1: 99.35 MiB, 104169472 bytes, 203456 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop2: 55.45 MiB, 58134528 bytes, 113544 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop3: 67.58 MiB, 70848512 bytes, 138376 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop4: 32.3 MiB, 33865728 bytes, 66144 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop5: 32.31 MiB, 33869824 bytes, 66152 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop6: 100.101 MiB, 105889792 bytes, 206816 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop7: 70.32 MiB, 73728000 bytes, 144000 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/nvme0n1: 232.91 GiB, 250059350016 bytes, 488397168 sectors
Disk model: Samsung SSD 970 EVO Plus 250GB          
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: BB4634C4-D47E-49AA-8AC6-E576A2308C1A

Device           Start       End   Sectors   Size Type
/dev/nvme0n1p1    2048   1050623   1048576   512M EFI System
/dev/nvme0n1p2 1050624   3147775   2097152     1G Linux filesystem
/dev/nvme0n1p3 3147776 488394751 485246976 231.4G Linux filesystem


Disk /dev/mapper/ubuntu--vg-ubuntu--lv: 115.71 GiB, 124222701568 bytes, 242622464 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sdb: 3.65 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: QuickPort Duo Cl
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 1FE11356-27C4-4CA4-9B4F-C41E4875C283

Device     Start        End    Sectors  Size Type
/dev/sdb1   2048 7814035455 7814033408  3.7T Microsoft basic data


Disk /dev/sda: 232.91 GiB, 250059350016 bytes, 488397168 sectors
Disk model: QuickPort Duo Cl
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 1FE11356-27C4-4CA4-9B4F-C41E4875C283

Device      Start       End   Sectors   Size Type
/dev/sda1      40    409639    409600   200M EFI System
/dev/sda2  409640 488392063 487982424 232.7G Microsoft basic data


Disk /dev/sdc: 1.84 TiB, 2000365289472 bytes, 3906963456 sectors
Disk model: My Passport 2626
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 192E2120-52C4-42F8-876B-92EB5E186258

Device      Start        End    Sectors  Size Type
/dev/sdc1      40     409639     409600  200M EFI System
/dev/sdc2  409640 3906961407 3906551768  1.8T Microsoft basic data


Disk /dev/loop8: 55.45 MiB, 58130432 bytes, 113536 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop9: 100.49 MiB, 105361408 bytes, 205784 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Something is not right with the /dev/mapper/ubuntu but not sure how to fix.

When I try to run sudo du -a /home | sort -n -r | head -n 20 to see where the biggest files are coming from I'm getting this output:

sort: write failed: /tmp/sortDfVeHU: No space left on device
90
0 + 10
Nate T avatar
it flag
Nate T
have you tried running bleachbit? It may correct the inconsistencies. I'm pretty sure it is available as an Apt pkg. Yea `sudo apt install bleachbit`. If it helps, let me know so I can add as an answer.
0
Reply
vanadium avatar
cn flag
vanadium
If `df` reports your root partition is full, then it is full. You will first need to free some space to enable the apt package management to do its work, so you can e.g. remove older kernels, limit your logs, etc. A possibility: destination was not mounted correctly, copied contents ended up in the mount point on your root partition. So that is the first place to go to look to delete files.
5
Reply
Soren A avatar
mx flag
Soren A
Could you please update the question with the exact `rsync` command you used.
0
Reply
cn flag
Ray
I *think* `rsync` downloads files by giving them a temporary file first and when they've been transferred, renames them to the correct name. Try going into the directory where you `rsync` to and do a `ls -al .*`. I think the temporary filename starts with a`.`. Then erase them.
0
Reply
us flag
sn0ep
Added my rsync command. In the output you'll see that my wdredpro is not mounted. That's because I remove the mount for now. But at the point of rsync WD red pro was mounted through fstab
0
Reply
cn flag
Rinzwind
to add to @vanadium full has TWO meanings: no more disk space OR no more inodes free. @ray is correct. `--temp-dir` to force rsync to use another dir.
1
Reply
us flag
sn0ep
For a default rsync installation where would the temp dir be located?
0
Reply
Charles Green avatar
cn flag
Charles Green
With no usb devices connected, check the contents of `/mnt/wdredpro`
1
Reply
us flag
sn0ep
I found that a huge portion of my NVME (drive where ubuntu runs on) was taken up by my unused docker images. So i have deleted all those that actually freed up 60+ Gigs of data. Which now atleast gives me some wiggle room to see where everything has gone wrong. Maybe ill investigate to have the docker images downloaded to my 4TB disk instead of the 120G nvme to avoid this issue in the future.
0
Reply
oldfred avatar
cn flag
oldfred
I had the same thing happen when I forgot to mount to location at same location as rsync command. It defaulted to location in /. I now have a check of mounts in my rsync script.
1
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.