I have a simple REST service set up that natively (standalone executable) handles 135k/rps.
Service is running on localhost:8181
autocannon benchmark running from separate machine yields:
┌───────────┬─────────┬─────────┬─────────┬─────────┬──────────┬──────────┬─────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼─────────┼─────────┼─────────┼─────────┼──────────┼──────────┼─────────┤
│ Req/Sec │ 58335 │ 58335 │ 109247 │ 135039 │ 106779.2 │ 18509.53 │ 58312 │
├───────────┼─────────┼─────────┼─────────┼─────────┼──────────┼──────────┼─────────┤
│ Bytes/Sec │ 9.74 MB │ 9.74 MB │ 18.3 MB │ 22.5 MB │ 17.8 MB │ 3.09 MB │ 9.74 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴──────────┴──────────┴─────────┘
When proxied with basic upstream setup through nginx the performance drops dramatically:
┌───────────┬─────────┬─────────┬─────────┬─────────┬──────────┬─────────┬─────────┐
│ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │
├───────────┼─────────┼─────────┼─────────┼─────────┼──────────┼─────────┼─────────┤
│ Req/Sec │ 13359 │ 13359 │ 14991 │ 19103 │ 15767.12 │ 1878.98 │ 13352 │
├───────────┼─────────┼─────────┼─────────┼─────────┼──────────┼─────────┼─────────┤
│ Bytes/Sec │ 2.53 MB │ 2.53 MB │ 2.83 MB │ 3.61 MB │ 2.98 MB │ 355 kB │ 2.52 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴──────────┴─────────┴─────────┘
Here are my nginx configurations (i have been experimenting with slightly, which resulted in very small improvements):
nginx.conf
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 768;
multi_accept on;
}
http {
open_file_cache max=200000 inactive=20s;
open_file_cache_valid 30s;
open_file_cache_min_uses 2;
open_file_cache_errors on;
access_log off;
error_log off;
sendfile on;
tcp_nopush on;
tcp_nodelay off;
keepalive_timeout 35;
types_hash_max_size 2048;
client_max_body_size 100M;
include /etc/nginx/mime.types;
default_type application/octet-stream;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
gzip off;
gzip_min_length 10240;
gzip_comp_level 1;
gzip_vary on;
gzip_disable msie6;
gzip_proxied expired no-cache no-store private auth;
server_tokens off;
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
sites-enabled/reverse-proxy.conf
upstream pindap_api {
least_conn;
server localhost:8181;
}
server {
listen 80;
server_name api.pindap;
access_log off;
error_log off;
location / {
proxy_buffering off;
proxy_pass http://pindap_api;
}
}
server {
listen 80;
server_name pindap;
access_log off;
error_log off;
location / {
proxy_buffering off;
proxy_pass http://localhost:8181;
}
}
What could be the cause of this? What else can I try?