Score:2

Server

Stop NGINX from locally caching a directory

Steven Seagul

10/24/22, 4:15 PM

So, the issue I am currently facing is that NGINX on my system keeps caching huge video files in memory when a HTTP request is made. This is causing NGINX to be using 20GB+ of memory and me having to run sync; echo 1 > /proc/sys/vm/drop_caches (clearing pagecache) frequently to stop constant swapping. I have provided below the relevant configuration.

nginx.conf

location /protected/ {
    internal;
    alias /usr/share/nginx/html/videos/;
}

script.php

function Download(){
    global $path, $fname;
    $file = "$path/$fname";
    header("Content-Type: video/mp4");
    header("Content-Length: " . filesize($file)); 
    header('Content-Disposition: attachment; filename="'.$fname.'"');
    header("X-Accel-Redirect: /protected/$fname");
    exit;
}

What I have tried in nginx.conf:

sendfile off;
if_modified_since off;
expires off;
etag off;
proxy_no_cache 1;
proxy_cache_bypass 1;
open_file_cache off;

P.S: I am calculating the memory usage for these files using pcstat and fincore, these MP4 files are being 100% cached when being watched by a user and each are 5GB+ in size.

341

1 + 1

memory

nginx

cache

php-fpm

Tero Kilkanen

10/24/22, 7:09 PM

Are you sure it is nginx using 20GB+ memory? Dropping page caches doesn't affect nginx memory usage, it just removes files from cache.

Score:1

Server

Tero Kilkanen

10/24/22, 7:13 PM

One can disable operating system caching for files served by nginx by using directio <size> setting. <size> is the threshold for file size, bigger files than that will be copied using DMA directly, and the files are not cached.

0 + 5

djdomi

10/24/22, 7:33 PM

proxy_no_cache 1; afaik on or off is correct

Tero Kilkanen

10/25/22, 6:43 AM

Not in this case since there is no reverse proxy configured. `proxy_no_cache` only affects reverse proxy caching.

djdomi

10/25/22, 9:45 AM

as you are not showing the full config we have to assum you use a proxy connection

djdomi

10/25/22, 9:48 AM

`expires -1;` disables any caching try as it invalids any requests and also `add_header 'Cache-Control' 'no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0';` force REALLY to ignore any caching behavior

Tero Kilkanen

10/25/22, 2:26 PM

That is disabling HTTP caching. It does not affect nginx memory usage in any way.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Stop NGINX from locally caching a directory

TH: หยุด NGINX จากการแคชไดเร็กทอรีในเครื่อง

RO: Opriți NGINX să memoreze local în cache un director

RU: Запретить NGINX локально кэшировать каталог

VI: Dừng NGINX lưu trữ cục bộ một thư mục

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.