Join Log in
Score:1
Dhananjay Bhardwaj avatar Server

Nginx - No last-modified or Etag response header on HTML

us flag
Dhananjay Bhardwaj

I am facing a rather odd issue with my web server. The server is configured with Ubuntu 20.04, Nginx v1.22.1, PHP 8.0, and MariaDB to host my WordPress website.

The issue is, the server is not sending any Etags or last-modified response headers for HTML pages, leading the browser to always get a 200 OK response, and never 304 Not Modified. I have configured browser caching with the ngx_http_headers_module module.

What's odd here is that both the Etags and last-modified headers, as well as the cache control headers are working flawlessly on all static assets, including JS, CSS, and images. But nothing for the HTML.

Below is my main config file for reference:

user jay;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 1024;
    multi_accept on;
}

http {

    ##
    # Basic Settings
    ##

    sendfile on;
    keepalive_timeout 60;
    tcp_nopush on;
    types_hash_max_size 2048;
    server_tokens off;
    client_max_body_size 64M;

    # server_names_hash_bucket_size 64;
    # server_name_in_redirect off;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ##
    # SSL Settings
    ##

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
    ssl_prefer_server_ciphers on;

    ##
    # Logging Settings
    ##

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    ##
    # Gzip Settings
    ##

    gzip on;

    # gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 5;
    # gzip_buffers 16 8k;
    # gzip_http_version 1.1;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    ##
    # Security
    ##
    add_header Strict-Transport-Security "max-age=31536000; includeSubdomains" always;
    add_header X-Xss-Protection "1; mode=block" always;
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header Referrer-Policy "strict-origin-when-cross-origin" always;

    ##
    # Virtual Host Configs
    ##

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;

    server {
        listen 80 default_server;
        listen [::]:80 default_server;
        server_name _;
        return 444;
    }
}

And below is my site-level config:

# Expires map
map $sent_http_content_type $expires {
    default                    off;
    text/html                  epoch;
    text/css                   max;
    application/javascript     max;
    ~image/                    max;
    ~font/                     max;
}

server {
    listen 443 ssl http2;

    server_name www.example.com;

    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

    access_log /home/jay/example.com/logs/access.log;
    error_log /home/jay/example.com/logs/error.log;

    expires $expires;

    root /home/jay/example.com/public/;
    index index.php;

    location / {
        try_files $uri $uri/ /index.php?$args;
    }

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass unix:/run/php/php8.0-fpm.sock;
        fastcgi_index index.php;
        include fastcgi_params;
    }
}

server {
    listen 443 ssl http2;

    server_name example.com;

    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

    return 301 https://www.example.com$request_uri;
}

server {
    listen 80;

    server_name example.com www.example.com;

    return 301 https://www.example.com$request_uri;
}

What I have done so far:

  • Eliminated the possibility of CloudFlare interfering with the response by completely removing it.
  • Removed all WordPress plugins as well as themes. Shifted to the default one.
  • Re-setup a new VPS with the same configuration.

But all in vain.

Can somebody please help? What could be causing this?

EDIT:

Here's the output for curl on the HTML:

HTTP/2 200 
server: nginx
date: Thu, 05 Jan 2023 14:59:14 GMT
content-type: text/html; charset=UTF-8
link: <https://www.example.com/wp-json/>; rel="https://api.w.org/"
x-fastcgi-cache: HIT
strict-transport-security: max-age=31536000; includeSubdomains
x-xss-protection: 1; mode=block
x-frame-options: SAMEORIGIN
x-content-type-options: nosniff
referrer-policy: strict-origin-when-cross-origin

The output stays the same even after repeated curl.

Below is the output for a static file, such the site's CSS:

HTTP/2 200 
server: nginx
date: Thu, 05 Jan 2023 15:01:50 GMT
content-type: text/css
content-length: 22256
last-modified: Wed, 04 Jan 2023 16:37:20 GMT
etag: "63b5ab40-56f0"
expires: Thu, 31 Dec 2037 23:55:55 GMT
cache-control: max-age=315360000
strict-transport-security: max-age=31536000; includeSubdomains
x-xss-protection: 1; mode=block
x-frame-options: SAMEORIGIN
x-content-type-options: nosniff
referrer-policy: strict-origin-when-cross-origin
accept-ranges: bytes
964
1 + 3
sv flag
Pothi Kalimuthu
As per https://nginx.org/r/etag , etag can be enabled only for static content. Not for dynamic HTML content. Something similar could be applicable for "last-modified" headers, I believe. Anyway, if your aim is to add expires headers, then there are other ways to achieve this, such as using a full page caching plugin where the static HTML is generated for each post / page. Nginx goes through the config in a phased manner, expires may be applied only on static resources. I may be wrong, though. Might need to use debug to understand when expires is applied to a request.
0
Reply
Dhananjay Bhardwaj avatar
us flag
Dhananjay Bhardwaj
@PothiKalimuthu Thanks for the explanation. In fact, I too have spiraled down to the same conclusion. After adding a page caching plugin to my WordPress installation, I can now see both ETags and last-modified response headers. Do you have any idea if it would be possible to add these same response headers on HTML documents cached by FastCGI cache? Since I am using page-caching at the server level, it feels redundant to repeat the same at application level (WordPress).
0
Reply
sv flag
Pothi Kalimuthu
No way to configure etag or last-modified header for content cached by FastCGI cache. Please see my answer to configure `expires`, though.
0
Reply
Score:1
avatar Server
sv flag
Pothi Kalimuthu

Welcome to ServerFault.

As per https://nginx.org/r/etag , etag can be enabled only for static content. Not for dynamic HTML content. Something similar could be applicable for "last-modified" headers, I believe. Anyway, if your aim is to add expires headers, then there are other ways to achieve this, such as using a full page caching plugin where the static HTML is generated for each post / page. Nginx goes through the config in a phased manner, expires may be applied only on static resources. I may be wrong, though. Might need to use debug to understand when expires is applied to a request.

Do you have any idea if it would be possible to add these same response headers on HTML documents cached by FastCGI cache?

No. There is no way to configure "etag" or "last-modified" headers for content cached by FastCGI cache.

If your aim is only to configure "expires", then you may use expires in the php location block like the following...

location ~* \.php$ {
    fastcgi_split_path_info ^(.+\.php)(/.*)$;
    if (!-f $document_root$fastcgi_script_name) { return 404; }

    # Mitigate https://httpoxy.org/ vulnerabilities
    fastcgi_param HTTP_PROXY "";

    include "fastcgi_params";
    fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name;
    fastcgi_index               index.php;
    fastcgi_pass                fpm;

    fastcgi_cache PHP_CACHE;
    fastcgi_cache_valid 200 10m;
    expires 10m;
    add_header "Cache-Control" "must-revalidate";
    add_header X-Cache $upstream_cache_status;

}

Here, "expires" works irrespective of FastCGI Cache. So, the above code will send cache-control: max-age=600 header to all PHP requests including to example.com/wp-login.php (even though it wouldn't be cached by FastCGI Cache). So, you may configure such URLs (/wp-admin/, /wp-comment-post.php, etc) in a different location block/s.

+ 0
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.