I have a Nginx reverse proxy, which mysteriously stops functioning after some hours. Error log gets filled with SSL errors connecting to upstream. Runing "nginx -s reload" (restarting nginx service) fixes the issue until next failure. Any ideas where to dig further?
Config files does not change, there are no any other activity in logs. Cannot pinpoint "time to fail", sometimes it seems to be 3-4 hours, sometimes somewhere around 8.
Application is some third-party Java Swing based dynamic web app from the legacy enterprise horror cellar... :)
Snippet for one of a bunch of the Nginx config Locations for the specific URI location. Cannot use one / location as one of the location needs MTLS authorization and others simply has to rely on whitelist.
location ~* ^/app-location(/?|/.*|-.*)$ {
include /etc/nginx/conf.d/include.APP.whitelist;
proxy_pass https://__APP_LB_DNS__:6443;
proxy_ssl_server_name on;
proxy_ssl_name __APP_LB_DNS__;
proxy_ssl_trusted_certificate __certificate-backend-ca-location__;
proxy_ssl_verify off;
proxy_ssl_verify_depth 3;
proxy_ssl_session_reuse on;
proxy_ssl_certificate __client-certificate-location__;
proxy_ssl_certificate_key __client-certificate-key-location__;
}
After some time I start to get 502: Bad Gateway errors and error.log is filled with such errors. 10.0.0.10 is the upstream load balancer IP for APP_LB_DNS hostname.
2022/02/09 21:36:45 [error] 12371#12371: *59560 SSL_do_handshake() failed (SSL: error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:SSL alert number 40) while SSL handshaking to upstream, client: CLIENT_IP, server: EXTERNAL_DNS_NAME, request: "GET /app-location/ HTTP/1.1", upstream: "https://10.0.0.10:6443/app-location/", host: "EXTERNAL_DNS_NAME", referrer: "http://CLIENT_DNS:53998/"
I have had to add options to ensure Nginx is communicating with Azure backend LB via its DNS name. Otherwise I was getting errors above constantly. Another thing that puzzles me is that it is only this weird app that requires such options. Other in house written apps does work well without those lines.
proxy_ssl_server_name on;
proxy_ssl_name __APP-DNS__;
Server is running Nginx on Centos 7 in an Azure VM
nginx-1.18.0-2.el7.ngx.x86_64
nginx-modsecurity3-centos7-3.0.4-1.x86_64