I have two Windows Servers 2019 with a round-trip time of 88 ms between them. When I try to measure bandwidth with iperf3 I get only around 16 Mbits/sec in both directions with a single stream. To get the actual bandwidth assigned of 1 Gbits/s I need to increase windows size to 1M and run it in 10 parallel streams using -w 1M -P 10. I also get good performance when I copy files between them.
Is there anything I can do to increase single stream performance? Here is the thing: Windows Server Backup is limited by this number. It takes some 2-4 hours for a simple backup to complete (copying the file of same size takes 15 minutes).
Here are the settings on both servers:
netsh int tcp show global
TCP Global Parameters
---------------------------------------------
Receive-Side Scaling State : enabled
Receive Window Auto-Tuning Level : normal
Add-On Congestion Control Provider : default
ECN Capability : enabled
RFC 1323 Timestamps : disabled
Initial RTO : 3000
Receive Segment Coalescing State : enabled
Non Sack Rtt Resiliency : disabled
Max SYN Retransmissions : 2
Fast Open : disabled
Fast Open Fallback : enabled
HyStart : enabled
Pacing Profile : off
Get-NetTCPSetting -SettingName Internet
SettingName : Internet
MinRto(ms) : 300
InitialCongestionWindow(MSS) : 10
CongestionProvider : CUBIC
CwndRestart : False
DelayedAckTimeout(ms) : 40
DelayedAckFrequency : 2
MemoryPressureProtection : Enabled
AutoTuningLevelLocal : Normal
AutoTuningLevelGroupPolicy : NotConfigured
AutoTuningLevelEffective : Local
EcnCapability : Enabled
Timestamps : Disabled
InitialRto(ms) : 3000
ScalingHeuristics : Disabled
DynamicPortRangeStartPort : 49152
DynamicPortRangeNumberOfPorts : 16384
AutomaticUseCustom : Disabled
NonSackRttResiliency : Disabled
ForceWS : Enabled
MaxSynRetransmissions : 2
AutoReusePortRangeStartPort : 0
AutoReusePortRangeNumberOfPorts : 0
Get-NetAdapterAdvancedProperty -Name External | Format-Table -Autosize
Name DisplayName DisplayValue RegistryKeyword RegistryValue
---- ----------- ------------ --------------- -------------
External Encapsulation Overhead 0 *EncapOverhead {0}
External Encapsulated Task Offload Enabled *EncapsulatedPacketTaskOffload {1}
External NVGRE Encapsulated Task Offload Enabled *EncapsulatedPacketTaskOffloadNvgre {1}
External VXLAN Encapsulated Task Offload Enabled *EncapsulatedPacketTaskOffloadVxlan {1}
External Flow Control Rx & Tx Enabled *FlowControl {3}
External Header Data Split Disabled *HeaderDataSplit {0}
External Interrupt Moderation Enabled *InterruptModeration {1}
External IPv4 Checksum Offload Rx & Tx Enabled *IPChecksumOffloadIPv4 {3}
External Jumbo Packet Disabled *JumboPacket {1514}
External Large Send Offload V2 (IPv4) Enabled *LsoV2IPv4 {1}
External Large Send Offload V2 (IPv6) Enabled *LsoV2IPv6 {1}
External Maximum Number of RSS Processors 16 *MaxRssProcessors {16}
External Preferred NUMA node 65535 *NumaNodeId {65535}
External Maximum Number of RSS Queues 2 Queues *NumRssQueues {2}
External Packet Priority & VLAN Packet Priority & VLAN Enabled *PriorityVLANTag {3}
External Receive Buffers 512 *ReceiveBuffers {512}
External Recv Segment Coalescing (IPv4) Enabled *RSCIPv4 {1}
External Recv Segment Coalescing (IPv6) Enabled *RSCIPv6 {1}
External Receive Side Scaling Enabled *RSS {1}
External Starting RSS CPU 0 *RssBaseProcNumber {0}
External Virtual Switch RSS Enabled *RssOnHostVPorts {1}
External RSS load balancing profile ClosestProcessor *RSSProfile {1}
External Speed & Duplex Auto Negotiation *SpeedDuplex {0}
External SR-IOV Enabled *SRIOV {1}
External TCP Checksum Offload (IPv4) Rx & Tx Enabled *TCPChecksumOffloadIPv4 {3}
External TCP Checksum Offload (IPv6) Rx & Tx Enabled *TCPChecksumOffloadIPv6 {3}
External Transmit Buffers 512 *TransmitBuffers {512}
External UDP Checksum Offload (IPv4) Rx & Tx Enabled *UDPChecksumOffloadIPv4 {3}
External UDP Checksum Offload (IPv6) Rx & Tx Enabled *UDPChecksumOffloadIPv6 {3}
External Virtual Machine Queues Enabled *VMQ {1}
External VXLAN UDP Destination Port Number 4789 *VxlanUDPPortNumber {4789}
External Wake on Magic Packet Enabled *WakeOnMagicPacket {1}
External Wake on Pattern Match Enabled *WakeOnPattern {1}
External DMA Coalescing Disabled DMACoalescing {0}
External Enable PME Disabled EnablePME {0}
External Interrupt Moderation Rate Adaptive ITR {65535}
External Log Link State Event Enabled LogLinkStateEvent {51}
External Locally Administered Address NetworkAddress {--}
External VLAN ID 0 VlanId {0}
External Wake on Link Settings Disabled WakeOnLink {0}
iperf3 -c address
Connecting to host address, port 5201
[ 4] local address port 57162 connected to address port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 768 KBytes 6.28 Mbits/sec
[ 4] 1.00-2.00 sec 2.00 MBytes 16.8 Mbits/sec
[ 4] 2.00-3.00 sec 2.12 MBytes 17.8 Mbits/sec
[ 4] 3.00-4.00 sec 2.00 MBytes 16.8 Mbits/sec
[ 4] 4.00-5.00 sec 2.00 MBytes 16.8 Mbits/sec
[ 4] 5.00-6.00 sec 2.00 MBytes 16.8 Mbits/sec
[ 4] 6.00-7.00 sec 2.00 MBytes 16.8 Mbits/sec
[ 4] 7.00-8.00 sec 2.12 MBytes 17.8 Mbits/sec
[ 4] 8.00-9.00 sec 2.00 MBytes 16.8 Mbits/sec
[ 4] 9.00-10.00 sec 2.12 MBytes 17.9 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 19.1 MBytes 16.0 Mbits/sec sender
[ 4] 0.00-10.00 sec 19.1 MBytes 16.0 Mbits/sec receiver
iperf3 -c address -w 1M -P 10 | Select-String 'SUM'
[SUM] 0.00-1.00 sec 16.1 MBytes 135 Mbits/sec
[SUM] 1.00-2.00 sec 44.1 MBytes 371 Mbits/sec
[SUM] 2.00-3.00 sec 105 MBytes 879 Mbits/sec
[SUM] 3.00-4.00 sec 109 MBytes 913 Mbits/sec
[SUM] 4.00-5.00 sec 110 MBytes 922 Mbits/sec
[SUM] 5.00-6.00 sec 93.1 MBytes 781 Mbits/sec
[SUM] 6.00-7.00 sec 96.0 MBytes 805 Mbits/sec
[SUM] 7.00-8.00 sec 98.2 MBytes 824 Mbits/sec
[SUM] 8.00-9.00 sec 101 MBytes 847 Mbits/sec
[SUM] 9.00-10.00 sec 102 MBytes 858 Mbits/sec
[SUM] 0.00-10.00 sec 874 MBytes 733 Mbits/sec sender
[SUM] 0.00-10.00 sec 874 MBytes 733 Mbits/sec receiver
