Score:0

Get real requester IP in containerized NGINX reverse proxy

ro flag

I have Docker Swarm stack with nginx as reverse proxy set up on OVH vps. I was trying to make use of allow/deny directives in location, but if I set deny all; it wouldn't work even for the ip's added with allow directive. After looking at access logs I found out, that all requests allegedly come from IP 10.0.0.2. Now I tried to get the actual IP first to at least be shown in logs, but with no luck. There is my nginx.conf:

events{}

http {

  map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
  }
  error_log /dev/stdout info;
  log_format json_combined escape=json
    '{'
      '"time":"$time_local",'
      '"httpRequest":{'
        '"requestMethod":"$request_method",'
        '"requestUrl":"$scheme://$host$request_uri",'
        '"requestSize":$request_length,'
        '"status":"$status",'
        '"responseSize":$bytes_sent,'
        '"userAgent":"$http_user_agent",'
        '"remoteIp":"$remote_addr",'
        '"serverIp":"$server_addr",'
        '"referer":"$http_referer",'
        '"latency":"${request_time}s",'
        '"protocol":"$server_protocol"'
      '}'
    '}';

  resolver 127.0.0.11 valid=30s;

  include /etc/nginx/mime.types;
  include /etc/nginx/sites-enabled/*.*;
}

proxy.conf:

set_real_ip_from        10.0.0.0/8;
real_ip_header          X-Forwarded-For;
real_ip_recursive       on;
proxy_set_header        X-Real-IP           $remote_addr;
proxy_set_header        X-Forwarded-For     $proxy_add_x_forwarded_for;
proxy_set_header        Host                $http_host;
proxy_set_header        X-Forwarded-Host    $http_host;
proxy_set_header        X-Forwarded-Proto   $scheme;
proxy_set_header        X-Forwarded-Port    $server_port;
proxy_set_header        Upgrade             $http_upgrade;
proxy_set_header        Connection          $connection_upgrade;
proxy_set_header        X-NginX-Proxy       true;
proxy_cache_bypass      $http_upgrade;
proxy_http_version      1.1;
proxy_read_timeout      20d;
proxy_buffering         off;
proxy_request_buffering off;
proxy_intercept_errors  on;
http2_push_preload      on;

and my location:

  location /api/ {
     allow XXX.XX.XX.X;
     deny  all;
     include /etc/nginx/proxy-options/proxy.conf;
     set $ocelot ocelot-service;
     proxy_pass http://$ocelot$uri$is_args$args;
     proxy_ssl_session_reuse off;
     proxy_redirect off;
     client_max_body_size 5M;
  }

What can I do so nginx logs actual IP of requester and if it's possible, to use the IP to compare with allow directive?

us flag
Have you checked that the connecting server sets the `X-Forwarded-For` header properly?
Pepsko avatar
ro flag
Honestly I don't know how I can check that
us flag
You need to find out that then. Check what is the software that connects to this nginx, and check its configuration.
Pepsko avatar
ro flag
Just to be sure - you mean software that sends request to the proxy, not the server hidden behind the proxy, right?
Pepsko avatar
ro flag
With further research I found out, that the problem is connected to nginx running inside container in docker swarm. I woul guess it's the docker swarm load balancer that sets the 10.0.0.2 IP. Still haven't found out how to fix this problem, as seems like the swarm load balancer doesn't set X-Forwarded-For header.
us flag
The IP address isn't "set". When TCP connection is opened, the source IP address is the one the connecting entity has. In this case, you need to see if Docker swarm can set the header.
Pepsko avatar
ro flag
Yeah it was just a mental shortcut. Anyways I managed to fix the problem - it was due to overlay network from Docker Swarm, I needed to run nginx service in host mode to resolve the problem. Anyways thanks for your time
Score:0
ro flag

Problem was caused due to the additional "overlay" network in docker swarm; I have bypassed this by using "host" networking.

services:
  nginx:
    ...
    ports:
      - target: 80
        published: 80
        protocol: tcp
        mode: host
      - target: 443
        published: 443
        protocol: tcp
        mode: host
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.