Score:2

Nginx how to disable per ip rate limiting

cn flag
Dan

I have a API which connect through private ip of the EC2 server and execute sequence of callbacks. I want disable per ip rate limiting on this scenario. I have tried this method in Nginx documentation.

This does not solved rate limit for issue. Access Log

192.168.192.51 - - [14/Jun/2021:00:09:55 +0530] "POST /project/api/v1/vendor/callback HTTP/1.1" 429 8576 "-" "Java/1.8.0_151" "-" "192.168.13.173" sn="192.168.13.173" rt=0.009 ua="unix:/var/run/php/php7.4-fpm.sock" us="429" ut="0.008" ul="8591" cs=-

Nginx conf file

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 1024;
    # multi_accept on;
}

http {

    geo $limit {
        default 1;
        192.168.192.51 0;
    }
 
    map $limit $limit_key {
            0 "";
            1 $binary_remote_addr;
    }
 
    limit_req_zone $limit_key zone=req_zone:10m rate=100r/s;

    ##
    # Basic Settings
    ##

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;
    # server_tokens off;

    # server_names_hash_bucket_size 64;
    # server_name_in_redirect off;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ##
    # SSL Settings
    ##

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
    ssl_prefer_server_ciphers on;

    ##
    # Logging Settings
    ##

    error_log /var/log/nginx/error.log warn;
    log_format main_ext '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for" ' '"$host" sn="$server_name" ' 'rt=$request_time ' 'ua="$upstream_addr" us="$upstream_status" ' 'ut="$upstream_response_time" ul="$upstream_response_length" ' 'cs=$upstream_cache_status' ;
    access_log /var/log/nginx/access.log main_ext;

    ##
    # Gzip Settings
    ##

    gzip on;

    # gzip_vary on;
    # gzip_proxied any;
    # gzip_comp_level 6;
    # gzip_buffers 16 8k;
    # gzip_http_version 1.1;
    # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    ##
    # Virtual Host Configs
    ##

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;

    fastcgi_buffers 8 16k;
        fastcgi_buffer_size 32k;
        fastcgi_connect_timeout 90;
        fastcgi_send_timeout 90;
        fastcgi_read_timeout 90;
}

Server Block

server {
    listen 80;
    listen 81;

        root /data/www;

        index index.html index.htm index.php;

        server_name 192.168.13.173;


        location / {
                try_files $uri $uri/ /index.php$is_args$args;
        }

    location /project{
                alias /data/www/project/public;
                try_files $uri $uri/ @project;

                location ~ \.php$ {
                        include snippets/fastcgi-php.conf;
                        fastcgi_param SCRIPT_FILENAME $request_filename;
                        fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
                }
        }

        location @project {
                rewrite /project/(.*)$ /project/index.php?/$1 last;
        }


    location ~ \.php$ {
                include snippets/fastcgi-php.conf;
                fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
        }

        location ~ /\.ht {
                deny all;
        }


}
djdomi avatar
za flag
If your Server gets Busy, that's a normal Reaction, did you tried `worker_connections 10240;` (the 10x value), however, look for https://www.nginx.com/blog/rate-limiting-nginx/ some limes maybe there some kind hidden
djdomi avatar
za flag
However, since a lot of API calls can be cached, i would suggest you to take a look at `set $memcached_key "$uri?$args"; memcached_pass host:11211; ` it may also improve your server also ;) a installed memcached is required for using this
Dan avatar
cn flag
Dan
Thanks, but I think something is wrong with this approach. Not a single change even after I added this rate limit white list. So I assume I'm missing something here?
sv flag
As per the access log provided in OP, the status code is shown as 429 (Too Many Requests). However, Nginx by default sends 503. Ref: https://nginx.org/r/limit_req_status Rate limiting becomes effective in Nginx only if both `limit_req_zone` and `limit_req` are configured. Your configuration is missing `limit_req` directive. My guess is that rate limiting isn't effective in Nginx or you haven't provided the full configuration. Please see the example configuration for your usecase... https://www.nginx.com/blog/rate-limiting-nginx/#Advanced-Configuration-Examples
Dan avatar
cn flag
Dan
You are correct @PothiKalimuthu, this was not generated from Nginx. This was applied from code (Laravel Framework). I was miss leaded by the access log. I thought this was generated from Nginx by looking at access log
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.