Score:0

Ubuntu

Files downloaded into external hard drive use up space in / on Ubuntu

Tomas Muller

2/8/24, 2:51 PM

I have a WD 2TB My Passport Portable External Hard Drive mounted on my Raspberry Pi 4 which I use to store files. I am currently using a Python script to download files locally into the external hard drive, but these are also using up space in my Raspberry. Oddly, I cannot find the files in the Raspberry.

Ubuntu 22.04 is installed in my Raspberry, using a 30GB SSD card. The command df -h gives the following output:

Filesystem      Size  Used Avail Use% Mounted on
tmpfs           380M  3,7M  376M   1% /run
/dev/mmcblk0p2   29G   17G   11G  62% /
tmpfs           1,9G     0  1,9G   0% /dev/shm
tmpfs           5,0M  4,0K  5,0M   1% /run/lock
/dev/mmcblk0p1  253M  125M  128M  50% /boot/firmware
tmpfs           379M   88K  379M   1% /run/user/1000
/dev/sda1       1,9T  327G  1,6T  18% /home/tomas/wdpassport

When I start downloading files into the external hard drive, they start using space in /dev/mmcblk0p2. They weird things is that this also happens when I connect the hard drive to my laptop (also with Ubuntu 22.04). I have tried mounting it on different directories as well, but this doesn't help. Any idea what is causing this?

1 + 8

external-hdd

Pilot6

2/8/24, 2:56 PM

It is unclear what kind of a script you are using, why do you need a script to download to an external drive, how do you know that "they start using space in /dev/mmcblk0p2"?

Tomas Muller

2/8/24, 3:02 PM

The script downloads FITS files using `astropy.table` with URLs. This runs on my laptop without problems. The issue comes when I download the files on my external hard drive. I know that the files are using space in my Raspberry because I run the command `df -h` again after downloading several files.

Pilot6

2/8/24, 3:03 PM

I have no idea what "FITS files" or "astropy.table" are. Probably the script writes to a wrong place. We can't tell anything without the script.

Tomas Muller

2/8/24, 3:06 PM

The FITS format is commonly used for astronomical images (not relevant as this also happens with other files such as png or jpg). The script is writing in the correct place as I can find the files in my external hard drive, where they should be. However, these are also using space in my Raspberry.

Pilot6

2/8/24, 3:07 PM

If they are "also using space" then the script writes to both drives.

Tomas Muller

2/8/24, 3:10 PM

That is the odd part. I know the files are only downloaded in the path given as I have tried this on different computers. Also, the downloaded files are only found in the external hard drive but cannot be found in the Raspberry.

Terrance

2/8/24, 3:21 PM

It might be possible that the script is downloading the file(s) to the RPi then moving it/them to the external keeping the deleted file(s) open wasting space on the RPi. You might want to see https://serverfault.com/questions/232525/df-in-linux-not-showing-correct-free-space-after-file-removal

Tomas Muller

2/8/24, 3:58 PM

Thanks. I have tried looking for deleted files, but I don't think this is the problem. The output of `lsof +L1` is `pipewire 1020 tomas 33u REG 0,1 2312 0 2053 /memfd:pipewire-memfd (deleted)`, `pulseaudi 1796 tomas 6u REG 0,1 67108864 0 2056 /memfd:pulseaudio (deleted)`.

Score:1

Ubuntu

Tomas Muller

2/8/24, 4:12 PM

I have found the source of the used space. The astropy package saves the cache in a hidden directory (/home/tomas/.astropy/cache in my case). That is why I wasn't finding it. I used the clear_download_cache() function of the package to clear the content and I got back the used space. Thanks for the help!

+ 1

Terrance

2/11/24, 3:38 PM

Nice find! Don't forget to mark you answer as correct. This can help others that might be experiencing the same thing. :)

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Files downloaded into external hard drive use up space in / on Ubuntu

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.