I'm running a javascript application with Prerender.io enabled for Google bots etc.
Using the configuration recommended by Prerender.io any matching user agents are redirected to their site and served the pre-rendered page.
There's a requirement to work with some partner services which will attempt to connect to our javascript pages and either make HTTP HEAD requests to get information about the page or use content negotiation to request some JSON. This should only happen for specific URLs, in particular those where there's a record ID, e.g. "/example.xyz890".
One problem with this is that the redirection to Prerender.io appears to happen before any rules matching the URL are checked. Can anyone think of a means of modifying this Nginx config so that I can check for specific URLs before any user agent matching and subsequent redirection takes place?
I've put an abridged copy of our config. below, with a location block in the port 443 section.
server {
listen 80;
server_name example.org;
return 301 https://$server_name$request_uri;
}
map $http_user_agent $prerender_ua {
default 0;
"~*Prerender" 0;
"~*googlebot" 1;
"~*yahoo!\ slurp" 1;
# Long list of user agents removed for brevity...
}
map $args $prerender_args {
default $prerender_ua;
"~(^|&)_escaped_fragment_=" 1;
}
map $http_x_prerender $x_prerender {
default $prerender_args;
"1" 0;
}
map $uri $prerender {
default $x_prerender;
"~*\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff|svg|eot)" 0;
}
server {
listen 443 ssl http2;
server_name example.org;
ssl_certificate /etc/ssl/example.org/fullchain.pem;
ssl_certificate_key /etc/ssl/example.org/privkey.pem;
ssl_protocols TLSv1.2 TLSv1.3;
# This is never reached if a user agent matches those listed for prerenderio.
location ~* "/example-([A-Za-z0-9]{4,6})$" {
add_header Link: "https://example.org/mysite-$1; rel='cite-as'";
# Detecting and responding (with a redirect) to a request for e.g. JSON would be useful here.
# JSON could be sent from api.example.org/example-$1, for example.
}
# Logs cut for brevity...
location /prerenderio {
if ($prerender = 0) {
return 404;
}
proxy_set_header X-Prerender-Token "abc123";
proxy_hide_header Cache-Control;
add_header Cache-Control "private,max-age=600,must-revalidate";
#resolve using Google's DNS server to force DNS resolution and prevent caching of IPs
resolver 8.8.8.8 8.8.4.4;
set $prerender_host "service.prerender.io";
proxy_pass https://$prerender_host;
rewrite .* /$scheme://$host$request_uri break;
}
}