Score:0

Nginx prerendering, redirections, content negotiation and HTTP HEAD requests

cn flag

I'm running a javascript application with Prerender.io enabled for Google bots etc.

Using the configuration recommended by Prerender.io any matching user agents are redirected to their site and served the pre-rendered page.

There's a requirement to work with some partner services which will attempt to connect to our javascript pages and either make HTTP HEAD requests to get information about the page or use content negotiation to request some JSON. This should only happen for specific URLs, in particular those where there's a record ID, e.g. "/example.xyz890".

One problem with this is that the redirection to Prerender.io appears to happen before any rules matching the URL are checked. Can anyone think of a means of modifying this Nginx config so that I can check for specific URLs before any user agent matching and subsequent redirection takes place?

I've put an abridged copy of our config. below, with a location block in the port 443 section.

server {
  listen 80;
  server_name example.org;
  return 301 https://$server_name$request_uri;
}


map $http_user_agent $prerender_ua {
    default       0;
    "~*Prerender" 0;

    "~*googlebot"                               1;
    "~*yahoo!\ slurp"                           1;
    # Long list of user agents removed for brevity...
}


map $args $prerender_args {
    default $prerender_ua;
    "~(^|&)_escaped_fragment_=" 1;
}


map $http_x_prerender $x_prerender {
    default $prerender_args;
    "1"     0;
}

map $uri $prerender {
    default $x_prerender;
    "~*\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff|svg|eot)" 0;
}

server {
    listen 443 ssl http2;
    server_name example.org;

    ssl_certificate /etc/ssl/example.org/fullchain.pem;
    ssl_certificate_key /etc/ssl/example.org/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;


    # This is never reached if a user agent matches those listed for prerenderio. 
    location ~* "/example-([A-Za-z0-9]{4,6})$" {
      add_header Link: "https://example.org/mysite-$1; rel='cite-as'";

      # Detecting and responding (with a redirect) to a request for e.g. JSON would be useful here. 
      # JSON could be sent from api.example.org/example-$1, for example. 
    }

    # Logs cut for brevity...

    location /prerenderio {
        if ($prerender = 0) {
            return 404;
        }

        proxy_set_header X-Prerender-Token "abc123";

        proxy_hide_header Cache-Control;
        add_header Cache-Control "private,max-age=600,must-revalidate";

        #resolve using Google's DNS server to force DNS resolution and prevent caching of IPs
        resolver 8.8.8.8 8.8.4.4;
        set $prerender_host "service.prerender.io";
        proxy_pass https://$prerender_host;
        rewrite .* /$scheme://$host$request_uri break;
    }
}
Score:1
cn flag

I think this solves it. The URL here is one where the record ID is simply an integer:

location ~* "/[0-9]+$" {
        proxy_http_version 1.1;
        add_header Link "https://www.example.org/$1; rel='cite-as'";
        if ($http_accept ~* 'application/json') {
            rewrite ^/([0-9]+) /content_negotiation/json/$1 break; 
            proxy_pass https://api.example.org;
        }
        root /var/www/example-client/dist;
        try_files $uri $uri/ /index.html;
    }

proxy_http_version was required to prevent curl complaining that the connection wasn't closed.

I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.