I'm seeing a strange issue with a wired connection to a router on the LAN, where TCP traffic is dropped and logged as a fragmented packet.
Scenario has MTU set at 1500 on network clients and on router ethernet adapter (according to ip addr show).
Example dropped packet message (broken into multi-line for readability):
May 12 09:44:05 kernel: DROP IN=eth0 OUT=tun11 MAC=<removed>
SRC=<desktop IP> DST=<external destination IP>
LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=54356 DF PROTO=TCP SPT=53370 DPT=443 SEQ=3216568911 ACK=0
WINDOW=64240 RES=0x00 SYN URGP=0 OPT (020405B40402080A002E76880000000001030307) MARK=0x8000000
From some light reading on TCP/IP I can see that opt 0204 refers to MSS and the size seems to be 1460, although I'm struggling to understand what the remaining bytes refer to (0402 080A 002E 7688 0000 0000 0103 0307). I can see no firewall rules on the router which applies a mark at all, but have limited detailed netfilter knowledge.
When I try calculating the max MTU size via ping -M do <router IP> -c 2 -s 1500 the client response is local error: message too long, mtu=1500 or when the ping cmd uses smaller MTU size to test, no response / all failed packets.
In the former I reduce the MTU by 28 and then get the latter scenario. I assume this means the MTU is appropriate but the packet is still being dropped. Example router log entry of this drop:
May 12 13:01:01 kernel: DROP IN=eth0 OUT= MAC=<removed>
SRC=<desktop IP> DST=<router IP>
LEN=1378 TOS=0x00 PREC=0x00 TTL=64 ID=0 DF
PROTO=ICMP TYPE=8 CODE=0 ID=5 SEQ=1 MARK=0x8000000
When I identify a more appropriate MTU using the ping test and reboot the desktop, packets continue to be fragmented and the ping test then returns a lower MTU (28 bytes less each time).
Desktop runs Kubuntu 20.04.4.
After some significant time trying to understand more about this scenario I tried adding the following iptables rules on the router:
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -m conntrack --ctstate DNAT -j ACCEPT
But this did not solve the problem.
This does not happen on Android devices connected to the same router via WiFi. The router is restarted overnight on a schedule and Xboxes encounter the same packet frag / drop scenario until they are restarted at least once. One XBox connects over WiFi, one wired to a router meshed to the problematic router.
Traffic between the WiFi router and the edge router appears to be unaffected - no sign of packets dropped due to fragmentation at that layer.
Workaround
I've discovered that running a traceroute from bash on the desktop somehow unjams the issue
Before and after this 'fix' the desktops LAN adapter is listed with the same MTU, whereas if MTU needed to be adjusted I would have expected this value to be changed (and I could then set it as a default):
eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
However I don't understand why the traceroute apparently fixes the issue. Any suggestions would be gratefully received at this point.
