Why we get TCP retransmissions through IPSec Site-by-Site Tunnel

gb flag

We use two pfsense firewalls at two locations and have connected the locations using IPsec site-by-site. That had worked well for a long time, now we had installed the 2.6.0 update at both locations and suddenly the speed dropped massively. I've analyzed this a bit and have read several posts about it.

The tunnel is online for phase 1 and phase 2, I can also ping the host in the other network and have no packet loss with ICMP ping. If I now measure with iperf from a host to the remote firewall, I have a bad throughput, only in the Kbit/s range and with Wireshark I see an awfully large number of TCP retransmissions, if I test from the other side then the speed is almost at normal level.

iperf result

With wireshark there are a lot of TCP - Retransmissions

wireshark analyze

I read on the internet that we should adjust the MTU and MSS on the PFsense, I tried that too and there is no change. Since it worked before the update, I don't really know what could be the reason and how I troubleshoot this issue.

UPDATE I created a trace on both firewalls and analyzed the packets. A packet arrives with errors on the remote firewall, but what exactly does that mean, or how can I determine what exactly went wrong?

In the following picture there is a snap of the trace with marked the origin packet.

source firewall dump

And in this picture you can see the same Packet on destination Firewall.

destination firewall dump


I found some more Informations, while I debugging this ipsec Tunnel I had found that the Packet Size is the Problem, some issues while fragmentation of Packets. If I do a ping (ping -f -l 969) with Packet Size of 969 Byte everythin is okay, with 970 there is packetloss.

So there is a Issue with fragmentation and I set the following Options in Firewall tab:

firewall tab


Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.