Score:0

iperf3 test bandwidth : TCP much slower than UDP

lr flag

I'm trying to find out why one of my servers, used for fileserving, is sending files at a very slow rate, around 7 Mbits/sec, through nginx.

I've been using iperf3 to test transfer rate between servers, both on Debian 11.

  • FrenchServ : $ iperf3 -s
  • CanadianServ : $ iperf3 -c 195.154.xxx.xxx -P 1 -t 10 -b 1G // Attempts to send, using 1 connection, in 10 sec, 1GB of data

This test over TCP gives the following results :

Connecting to host 195.154.xxx.xxx, port 5201
[  5] local 142.44.xxx.xxx port 60610 connected to 195.154.xxx.xxx port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  1.04 MBytes  8.69 Mbits/sec    0    125 KBytes       
[  5]   1.00-2.00   sec   768 KBytes  6.29 Mbits/sec    0    125 KBytes       
[  5]   2.00-3.00   sec   768 KBytes  6.29 Mbits/sec    0    125 KBytes       
[  5]   3.00-4.00   sec   640 KBytes  5.24 Mbits/sec    0    125 KBytes       
[  5]   4.00-5.00   sec   768 KBytes  6.29 Mbits/sec    0    125 KBytes       
[  5]   5.00-6.00   sec   768 KBytes  6.29 Mbits/sec    0    125 KBytes       
[  5]   6.00-7.00   sec   768 KBytes  6.29 Mbits/sec    0    125 KBytes       
[  5]   7.00-8.00   sec   640 KBytes  5.24 Mbits/sec    0    125 KBytes       
[  5]   8.00-9.00   sec   768 KBytes  6.29 Mbits/sec    0    125 KBytes       
[  5]   9.00-10.00  sec   768 KBytes  6.29 Mbits/sec    0    125 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  7.54 MBytes  6.32 Mbits/sec    0             sender
[  5]   0.00-10.09  sec  7.08 MBytes  5.89 Mbits/sec                  receiver

iperf Done.

It's very slow, I may stack connections to have faster bandwidth, but would like to keep it 1 connection.

Testing over UDP now.

  • CanadianServ : $ iperf3 -c 195.154.xxx.xxx -P 1 -t 10 -b 1G -u

This outputs:

Connecting to host 195.154.xxx.xxx, port 5201
[  5] local 142.44.xxx.xxx port 37350 connected to 195.154.xxx.xxx port 5201
[ ID] Interval           Transfer     Bitrate         Total Datagrams
[  5]   0.00-1.00   sec   113 MBytes   950 Mbits/sec  81331  
[  5]   1.00-2.00   sec   114 MBytes   955 Mbits/sec  81773  
[  5]   2.00-3.00   sec   114 MBytes   956 Mbits/sec  81887  
[  5]   3.00-4.00   sec   114 MBytes   957 Mbits/sec  81904  
[  5]   4.00-5.00   sec   114 MBytes   956 Mbits/sec  81862  
[  5]   5.00-6.00   sec   114 MBytes   957 Mbits/sec  81913  
[  5]   6.00-7.00   sec   114 MBytes   957 Mbits/sec  81909  
[  5]   7.00-8.00   sec   114 MBytes   957 Mbits/sec  81915  
[  5]   8.00-9.00   sec   114 MBytes   956 Mbits/sec  81880  
[  5]   9.00-10.00  sec   114 MBytes   957 Mbits/sec  81904  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.00  sec  1.11 GBytes   956 Mbits/sec  0.000 ms  0/818278 (0%)  sender
[  5]   0.00-10.41  sec   600 MBytes   484 Mbits/sec  0.029 ms  387089/818278 (47%)  receiver

iperf Done.

UDP actual bandwidth tops around 500 Mbits/sec, which is perfectly fine given both server's network interfaces.

How can I make TCP transfer use more bandwidth ?? I've tried on my local computer, I get around 30 Mbits/sec on TCP transfers.

br flag
What's the spec of the server?
Baptiste Placé avatar
lr flag
FrenchServ: Intel(R) Xeon(R) CPU E3-1220 V2 @ 3.10GHz - 16 GB RAM CanadianServ: Intel Xeon-E 2136 - 6c/12t - 3.3GHz/4.5GHz - 32GB RAM
br flag
Why so old - that's a decade old CPU?
Baptiste Placé avatar
lr flag
Just one test server, go the exact same speed on a newer spec
Score:0
lr flag

Found the solution in the server configuration, TCP Window was disabled as recommended for standard Apache configuration (described here)

$ vi /etc/sysctl.conf

# Set this value and save
net.ipv4.tcp_window_scaling = 1

# check with
$ sysctl -p

Now we need to adjust the TCP Window value, as explained here or here.

I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.