uWSGI aborts responses, loses connection to nginx

Felix

1/21/23, 8:59 AM

I'm running a small web application written in Python, running in uWSGI and served through nginx. There's a component that generates ZIP files for downloading, which occasionally can be quite large (several GB). It often happens that the connection between nginx and uWSGI is broken and the request is aborted; nginx ignores the truncated response while the browser runs into a timeout because it keeps the connection opened, expecting more response data. The application generates a proper Content-Length header.

From the uWSGI log:

uwsgi_response_write_body_do(): Broken pipe [core/writer.c line 429] during GET [...]
OSError: write error
SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request [...] !!!

I've already set socket-timeout, socket-send-timeout and socket-write-timeout to 180 in the uWSGI configuration, to no avail. The nginx conf includes uwsgi_read_timeout 180s; and uwsgi_buffering off;

The effect is mostly reproducible, in that it happens most of the time, especially with large responses, but never at the same offset. Repeating the request over and over again might eventually lead to completion.

584

0 + 0

http

python

nginx

uwsgi

Michael Hampton

1/21/23, 11:05 AM

This seems more like an app design problem. Web requests really aren't intended to sit there for several minutes waiting for a response. You should rather start a background job to create the ZIP file and then give the user a URL where they can check its status or pick up the file when it is complete.

Felix

1/21/23, 11:12 AM

@MichaelHampton The file is generated on the fly. Response delay is typically less than 10 seconds, but delivery of the response body takes time. The connection is aborted while the response is already being sent to the client. The user agent reports a download progress of sometimes 20%, sometimes 80%, varying a lot, then the data stops flowing and at some point, the user agent reports a timeout while downloading.

Michael Hampton

1/21/23, 11:14 AM

Hmm. You should check the nginx error log.

Felix

1/21/23, 11:18 AM

Nothing in the nginx error log; the access log only reports the requests with the truncated length, ignoring the fact that Content-Length suggest that the connection was prematurely interrupted.

Michael Hampton

1/21/23, 11:19 AM

OK, so here's an interesting question. How can you possibly know the length of the compressed file before you've compressed it?

Felix

1/21/23, 11:25 AM

I don't compress the files, since they're usually already compressed (mostly JPEG files), so DEFLATE will not gain much space anyway. The files are stored without compression. I get a list of files to send, and from their file sizes, I calculate how big the zip file will be precisely. That's reported as Content-Length, then the ZIP file is generated while being sent to the client. I tested this a lot, and the length calculation is not the problem.

Score:0

Server

Felix

1/30/23, 10:12 PM

Turned out neither my application nor nginx was the problem, but a faulty packet filter in front of both.

0 + 0

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: uWSGI aborts responses, loses connection to nginx

TH: uWSGI ยกเลิกการตอบกลับ สูญเสียการเชื่อมต่อกับ nginx

RO: uWSGI anulează răspunsurile, pierde conexiunea la nginx

RU: uWSGI прерывает ответы, теряет соединение с nginx

VI: uWSGI hủy phản hồi, mất kết nối với nginx

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.