Join Log in
Score:0
matisa avatar Server

Two directories share storage without symbolic link

cn flag
matisa

I am in a pickle.

I have a server (physicals machine) that has a link between a directory and a partition. I don't see a symbolic link.

The problem is that the root partition shows 0% space. Even if I delete data from other directories it doesn't effect the root partition correctly.

I deleted 200MB and it showed 2MB of free space.
There is nothing in fastab.

The link is between /path/backups/ftp to /home/ftp/public_html

This is my df -h

devtmpfs                  63G     0   63G   0% /dev
tmpfs                     63G     0   63G   0% /dev/shm
tmpfs                     63G  4.1G   59G   7% /run
tmpfs                     63G     0   63G   0% /sys/fs/cgroup
/dev/mapper/centos-root   45G   45G  208K 100% /
/dev/sda1               1014M  194M  821M  20% /boot
/dev/sdb                 4.3T   89M  4.1T   1% /path/vms
/dev/sdc                  11T  7.5T  2.9T  73% /path/backups
tmpfs                     13G     0   13G   0% /run/user/0

this is fastab file:

#
# /etc/fstab
# Created by anaconda on Tue Nov 17 22:49:51 2020
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/centos-root / xfs rw,seclabel,relatime,attr2,inode64,noquota 0 0
UUID=dca34673-80bb-4c10-a1ca-cd76167ebcf4 /boot                   xfs     defaults        0 0
/dev/mapper/centos-swap swap                    swap    defaults        0 0
/dev/sdc        /path/backups    ext4    defaults    0 2
/dev/sdb        /path/vms        ext4    defaults    0 2

This is my lsblk:

NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0   50G  0 disk
├─sda1            8:1    0    1G  0 part /boot
└─sda2            8:2    0   49G  0 part
  ├─centos-root 253:0    0   45G  0 lvm  /
  └─centos-swap 253:1    0    4G  0 lvm  [SWAP]
sdb               8:16   0  4.3T  0 disk /path/vms
sdc               8:32   0 10.9T  0 disk /path/backups

I thought maybe it can be a hard link, but I can't find any proof of that, and also it should not be possible to hard link 2 directories.

Any ideas?

Edit:

The mtab contenct:

sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
devtmpfs /dev devtmpfs rw,nosuid,size=65906808k,nr_inodes=16476702,mode=755 0 0
securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run tmpfs rw,nosuid,nodev,mode=755 0 0
tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0
cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0
pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_prio,net_cls 0 0
cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0
cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0
cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpuacct,cpu 0 0
cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
configfs /sys/kernel/config configfs rw,relatime 0 0
/dev/mapper/centos-root / xfs rw,relatime,attr2,inode64,noquota 0 0
systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=25,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=12844 0 0
debugfs /sys/kernel/debug debugfs rw,relatime 0 0
mqueue /dev/mqueue mqueue rw,relatime 0 0
hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0
/dev/sda1 /boot xfs rw,relatime,attr2,inode64,noquota 0 0
/dev/sdb /path/vms ext4 rw,relatime,data=ordered 0 0
/dev/sdc /path/backups ext4 rw,relatime,data=ordered 0 0
sunrpc /var/lib/path/rpc_pipefs rpc_pipefs rw,relatime 0 0
nfsd /proc/fs/nfsd nfsd rw,relatime 0 0
/dev/sdc /home/ftp/public_html ext4 rw,relatime,data=ordered 0 0
binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0
tmpfs /run/user/0 tmpfs rw,nosuid,nodev,relatime,size=13183736k,mode=700 0 0

I have /dev/sdc mounted twice. It makes stuff work funny.

I see this in this lines:

/dev/sdc /path/backups ext4 rw,relatime,data=ordered 0 0
/dev/sdc /home/ftp/public_html ext4 rw,relatime,data=ordered 0 0

The storage problem and also I see that it shows 97% use of inodes, even though there is not so many files actually there.

63
0 + 0
Nikita Kipriyanov avatar
za flag
Nikita Kipriyanov
Directories can't be hardlinked, period. Also show us `mount`, there could be *bind mounts* that actually really allow share storage without links; however, I suspect you just faced with ordinary file system overflow, nothing special. This also may be caused by lost inodes; perform a file system check (this is root fs, you will need to reboot the machine; assure it has enoungh space to return). This may be caused by deleted files still opened (so there is no name, but there is still an inode), to release them, kill processes that are keeping them.
2
Reply
Nikita Kipriyanov avatar
za flag
Nikita Kipriyanov
By the way, there could not be *any* hard link between anything in /path/backups and /, because those are *different* file systems. Hard linking is only possible within a single file system.
0
Reply
matisa avatar
cn flag
matisa
Yes! Thank you. There is a mount to 2 different directories. I am not sure how to proceed without losing data.
0
Reply
Nikita Kipriyanov avatar
za flag
Nikita Kipriyanov
Don't panic. This is exactly how a bind mount looks. This can be frustrating, but first check `findmnt` (check https://unix.stackexchange.com/questions/295525/how-is-findmnt-able-to-list-bind-mounts/346460). Also, this has nothing to do with your root file system overflow. Investigate that with `du -sh *` descend, as the answer below suggests.
0
Reply
matisa avatar
cn flag
matisa
Actually I fixed it. I unmounted the mount, deleted all the files that took space. AS I suspected there were files on the original path. Restarted the machine which freed all the old inodes and remounted the path (As we still need the mount there). Thanks a lot. Please add this as an answer so I can except it.
0
Reply
Score:-1
Zareh Kasparian avatar Server
us flag
Zareh Kasparian

why don't you use "du" instead of "df". Basically, df reads the superblock only. du reads each object and sums them up. so you may use "du -sh" to find the size of each directory with all its contents inside. Also you may have a look at du vs. df difference

0 + 0
matisa avatar
cn flag
matisa
Thanks. I used du... it doesn't help me to know the directory size, or the object size. That is not the problem. The problem is that there is some weird link between 2 directories that effect the whole partition in a way that is very problematic.
0
Reply
Zareh Kasparian avatar
us flag
Zareh Kasparian
whatever it is, du command can report it back to you. you just need to traverse to the directories step by step to find the desired file.
0
Reply
matisa avatar
cn flag
matisa
I have a very good understanding of the du command. I did use it to locate data I can delete outside of the linked directories. I deleted over 300 MB now. It doesn't help. What I need is to locate the type of link that links this paths and fix this.
0
Reply
Nikita Kipriyanov avatar
za flag
Nikita Kipriyanov
Why are you so sure there are some mysterious "links"? And why do you think this will ever help you? Don't waste time to that, search for large directories with `du -sh *`, descending from the root down to largest directories, often `/home` and `/var`.
1
Reply
matisa avatar
cn flag
matisa
Look at the correspondence on the original question. You will see what happened.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.