Nginx UDP proxy shared connection is busy

We are proxying UDP packets for a game server through a Nginx reverse proxy. Clients timeout very often because the Nginx can't handle the requests. Initially it works flawlessly, but when more clients are connected to the game server (40-50), we encounter problems.

How could we make Nginx handle it?

Here is the proxy setup

stream {
    upstream backend {
        server [server-ip]:[port];
    }

    server {
        listen [port];
        proxy_pass backend;
    }

    server {
        listen [port] udp reuseport;
        proxy_pass backend;
    }
}

Our error log is full of errors like this:

2023/02/18 12:40:29 [alert] 3951459#3951459: *5777636 shared connection is busy while proxying and sending to client, udp client: [client-ip], server: 0.0.0.0:[port], upstream: "[server-ip]:[port]", bytes from/to client:1684904/1724744, bytes from/to upstream:1725538/1684904

How can I configure Nginx to handle these UDP packets?

Try this:

stream { upstream backend { server [server-ip]:[port]; }

server {
    listen [port];
    proxy_pass backend;
}

server {
    listen [port] udp reuseport;
    proxy_pass backend;
    proxy_bind $remote_addr transparent;
}

}