Score:0

Logs hidden by a mounted disk in the same directory

in flag

We have (more than) a linux server on AWS that use a network file system, in our case AWS EFS, mounted on /logs directory.

It happens sometimes that:

  • the machine is rebooted
  • for any reason the network file system is not mounted, so all process starts to log in /logs but on the main (system) partition
  • disk space quickly starts to be low and, in worst case, the disk become completely full.

While debugging the issue a reboot happened and network file system comes back to normal so that my server can mount the disk and starts logging in the correct partition.

BUT the logs generated while network file system are still on the main partition, they are NOT accessible since a disk is mounted in the same /logs directory and they use disk space that cannot be reclaimed.

Is there any way apart unmount the network file system to access that logs so that I can trash or move them to the correct location avoiding to use precious space in system partition?

Score:2
cn flag

You can recover the underlying filesystem by utilizing a bind mount and the fact that bind mounts are not recursive by default.

mkdir /mnt/var
mount --bind /var /mnt/var

Now the underlying files should be visible under /mnt/var/log. Unless /var/log was a separate mount already.

If /var/log was a separate mount before the overmount, then you can just mount the original mount somewhere else.

You can use findmnt /my/mountpoint to detect if something is a mount and from where.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.