Score:1

Node on Nginx - proxy_request_buffering off doesn't work with https? Super slow https image uploads

us flag

I have a node server running on nginx as a reverse proxy, which is supposed to handle image uploads. Its ssl cert and domain are configured through cloudflare.

Aside from setting the proxy_pass to pass 443 requests to the node on localhost, I haven't changed the default configuration much. The more notable nginx directives I have set at the bottom of the http block are:

# Disabled nginx testing for overweight requests - handled in node
client_max_body_size 0;     

# Request and response buffering disabled - to try and be able to reject overweight requests in node ASAP 
proxy_request_buffering off;   
proxy_buffering off;    

# Enabled to avoid buffering http 1.1 chunked if sent 
proxy_http_version 1.1;     

# Disabled writing responses to a file 
proxy_max_temp_file_size 0; 

Regardless of the client used - postman/insomnia/axios - when sending form data requests with a 4MB image as one of the properties through http, the request reaches the first node middleware after approximately 100ms, which is great.However, when sending the same request over https, it takes approximately 4 seconds to reach that first node middleware. 4 seconds being approximately the time needed to send over the entirety of the request (file).

When sending GET requests to a simple endpoint returning a 200 OK response, http and https calls take a similar amount of time to resolve - within 100ms from each other- with http being slightly faster.That leads me to believe that the huge overhead observed when sending form data requests is caused by the https request not being passed to the node server immediately as it's being received by nginx.

When I enable nginx client_max_body_size directive and set it to i.e. 256K. The same problem arises - http call is rejected with a 413 response from nginx after a reasonable ~100ms, while https request takes around 4 seconds until 413 rejection.

I have tested a bunch of different directive configurations but nothing affected https upload speed in a significant manner. At this point I'm running out of ideas and would appreciate any pointers as to what can cause that and what the solution would be. Cheers!

Score:0
za flag
  1. proxy buffering works with just any scheme.

  2. proxy buffering usually speeds the whole things up, not slows it down.

    So I suppose you're blaming the wrong cause.

  3. 4 megabyte image size is generally not an image size that fits the whole idea of "images on the public Internet", unless you're hosting a NASA site with James Webb hi-resolution photos, so scale them down and make sure you're using compression (either sening JPEgs or enabling compression for non-compressed raster formats).

  4. I'd also make sure the reverse proxy you're operating is using a cache so it doesn't redownload these gigantic pics each time they are requested (and out-of-the-box nginx comes with a disabled one).

intern3t-us3r avatar
us flag
Hey there, thanks for responding :) ad 1,2. What I meant is that proxy_request_buffering visibly affects the time needed for node to start handling request (fire middleware) sent over http but has no visible effect for https. ad 3,4. The images are only being uploaded so caching won't help here. 4MB isn't my target image size, it's just used as an example of a large request I want to reject as soon as possible but am unable to when sent over https. I also discovered that ditching nginx and using node's https module gave the same results - https being handled only after few seconds pass
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.