I am actually working in a company and to improve SEO, i am trying to setup our angular (10) web app with prerender.io to send rendered html to crawlers visiting our website.
The app is dockerized and exposed using an nginx server. To avoid conflict with existing nginx conf (after few try using it), i (re)started configuration from the .conf file provided in the prerender.io documentation (https://gist.github.com/thoop/8165802) but impossible for me to get any response from the prerender service.
I am always facing: "502: Bad Gateway" (client side) and "could not be resolved (110: Operation timed out)" (server side) when i send a request with Googlebot as User-agent.
After building and running my docker image, the website is correctly exposed on port 80. It is fully accessible when i use a web browser, but the error occurs when i try a request as a bot (using curl -A Googlebot http://localhost:80).
To verify if the prerender service correctly receive my request when needed i tried to use an url generated on pipedream.com, but the request never comes.
I tried using different resolver (8.8.8.8 and 1.1.1.1) but nothing changed.
I tried to increase the resolver_timeout to let more time but still the same error.
I tried to install curl in the container because my image is based on an alpine image, curl was successfully installed but nothing changed.
Here is my nginx conf file :
server {
listen 80 default_server;
root /usr/share/nginx/html;
index index.html;
location / {
try_files $uri @prerender;
}
location @prerender {
proxy_set_header X-Prerender-Token TOKEN_HERE;
set $prerender 0;
if ($http_user_agent ~* "googlebot|bingbot|yandex|baiduspider|twitterbot|facebookexternalhit|rogerbot|linkedinbot|embedly|quora link preview|showyoubot|outbrain|pinterest\/0\.|pinterestbot|slackbot|vkShare|W3C_Validator|whatsapp") {
set $prerender 1;
}
if ($args ~ "_escaped_fragment_") {
set $prerender 1;
}
if ($http_user_agent ~ "Prerender") {
set $prerender 0;
}
if ($uri ~* "\.(js|css|xml|less|png|jpg|jpeg|gif|pdf|doc|txt|ico|rss|zip|mp3|rar|exe|wmv|doc|avi|ppt|mpg|mpeg|tif|wav|mov|psd|ai|xls|mp4|m4a|swf|dat|dmg|iso|flv|m4v|torrent|ttf|woff|svg|eot)") {
set $prerender 0;
}
#resolve using Google's DNS server to force DNS resolution and prevent caching of IPs
resolver 8.8.8.8;
resolver_timeout 60s;
if ($prerender = 1) {
#setting prerender as a variable forces DNS resolution since nginx caches IPs and doesnt play well with load balancing
set $prerender "service.prerender.io";
rewrite .* /$scheme://$host$request_uri? break;
proxy_pass http://$prerender;
}
if ($prerender = 0) {
rewrite .* /index.html break;
}
}
}
And here is my Dockerfile:
FROM node:12.7-alpine AS build
ARG environment=production
WORKDIR /usr/src/app
COPY package.json package-lock.json ./
RUN npm install
COPY . .
RUN npm run build -- --configuration $environment
# Two stage build because we do not need node-related things
FROM nginx:1.17.1-alpine
RUN apk add --no-cache curl
COPY --from=build /usr/src/app/dist/app /usr/share/nginx/html
COPY prerender-nginx.conf /etc/nginx/conf.d/default.conf
EXPOSE 80
Hoping that you could help me :)