I setup a Nginx Proxy in front of my S3 object storage and I'm able to download files without any issues so far.
However, if I try to download the same file from, let's say, 30 different servers, at the same time, only the first 10-11 servers are able to continue downloading while the rest gets cancelled with the following error;
gzip: stdin: unexpected end of file
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
This is the command I'm running to download the file;
sudo curl -f -Ls https://cdn.mydomain.com/files/somefile.tar.gz | tar xvz
And this is my nginx config file under /etc/nginx/sites-enabled/cdn.conf
;
server {
listen 80;
server_name cdn.mydomain.com;
if ($host != "cdn.mydomain.com") {
return 444;
}
access_log /var/log/nginx/reverse-access.log;
error_log /var/log/nginx/reverse-error.log;
location /files {
proxy_pass https://myBucket.s3.nl-ams.scw.cloud/files;
proxy_set_header X-Real-IP $remote_addr;
}
}
My nginx.conf
is the default file that gets generated as I first installed the nginx on my server.
I don't know what's limiting this but how can I allow more concurrent downloads in this setup?