Score:3

nginx config for reverse proxying UDP traffic (minecraft app) - 90 message too long

kr flag
seb

Trying to proxy udp traffic. nginx doesn't throw any error about the configuration. The client connects until half-way (it says it could reach the end server) but the connection then gets stuck and eventually closes out with timeout.

nginx version: 1.21.3 OS: Ubuntu 18.04

nginx.conf:

worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

worker_rlimit_nofile 30000;

events {
    worker_connections 30000;
    multi_accept on;
}

stream{
server {
    listen       *:4800-4899 udp;
    proxy_pass   217.178.x.x:$server_port;
}
}

error.log:

2213#2213: *3 recv() failed (90: Message too long) while proxying and reading from upstream, udp client: 49.98.x.x, server: 66.42.x.x:4801, upstream: "217.178.x.x:4801", bytes from/to client:1464/0, bytes from/to upstream:0/1464

49.98.x.x: client IP 66.42.x.x: proxy IP 217.178.x.x: end server IP

ip a output

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever

2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 56:00:03:95:cc:59 brd ff:ff:ff:ff:ff:ff
    inet 66.42.x.x/23 brd 66.42.x.255 scope global dynamic enp1s0
       valid_lft 57402sec preferred_lft 57402sec
    inet6 fe80::5400:3ff:fe95:[xxx]/64 scope link 
       valid_lft forever preferred_lft forever

Any idea what's wrong? Can anyone see any issue in the config?

Thanks

Michael Hampton avatar
cz flag
I see no issue with your nginx configuration, but there is possibly an issue with the server's network configuration. Please post the output of `ip address` and note which interface routes toward the Internet and which (if any) goes to the machine running your backend application. Also see our [reference](https://meta.serverfault.com/q/963/126632) on obfuscating IP addresses (i.e. generally you should not).
seb avatar
kr flag
seb
Thanks, I've added the ip a output. There is no private IP on the proxy.
Michael Hampton avatar
cz flag
Interesting. You don't seem to have an MTU issue. 1464 bytes ought to be fine, unless there is some gap in my knowledge! There probably is. I've upvoted your question as it is a good and interesting one and should have more attention on it.
shunya avatar
gn flag
I am also getting the exact same error. It's been over a year since the post, but is there any solution?
Score:2
fr flag

Perhaps the problem "90: Message too long" can be associated with the EMSGSIZE error (If the message is too long to pass atomically through the underlying protocol, the error EMSGSIZE is returned, and the message is not transmitted), in which case you can try to solve it by increasing the size of the socket send buffer. The following parameters have standard values:

net.core.wmem_default = 212992
net.core.wmem_max = 212992

The current values ​​on your server can be checked with the command:

sysctl -a | grep "net.core.wmem"

For the test, using the following command, you can set, for example, the following values:

sysctl -w net.core.wmem_default = 9999999
sysctl -w net.core.wmem_max = 9999999
echo "net.core.wmem_default = 9999999
net.core.wmem_max = 9999999 ">> /etc/sysctl.conf

Next, check that they were applied correctly (now must be 9999999):

sysctl -a | grep "net.core.wmem"

And try to reproduce the problem.

seb avatar
kr flag
seb
Thanks. The values were the default ones. I changed them to 9999999, checked they were applied correctly, restarted nginx but issue is still exactly the same, same error in log.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.