With default MTU as shown below:
hosta$ ifconfig eth0 | grep mtu
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
I can ping from a different server using a 1500 payload
hostb$ ping -s 1500 -c 2 hosta
PING hosta (hosta) 1500(1528) bytes of data.
1508 bytes from hosta: icmp_seq=1 ttl=64 time=0.273 ms
1508 bytes from hosta: icmp_seq=2 ttl=64 time=0.314 ms
--- hosta ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1025ms
rtt min/avg/max/mdev = 0.273/0.293/0.314/0.020 ms
tcpdump on hosta is all good
12:01:40.237047 IP hostb > hosta: ICMP echo request, id 3052, seq 1, length 1480
12:01:40.237048 IP hostb > hosta: icmp
12:01:40.237116 IP hosta > hostb: ICMP echo reply, id 3052, seq 1, length 1480
I can lower the MTU on hosta up to 1488 and the 1500 payload ping works.
The magic number is MTU=1487
hosta $ ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1487
<snip>
When I send the 1500 byte ping (not messing with fragmentation flag) there is no response
hosb $ ping -s 1500 -c 2 hosta
PING hosta (hosta) 1500(1528) bytes of data.
--- hosta ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1032ms
tcpdump on hosta looks like this:
12:01:07.421196 IP hostb > hosta: icmp
12:01:08.443698 IP hostb > hosta: icmp
hosta shows
net.ipv4.ip_forward_use_pmtu = 0
net.ipv4.ip_no_pmtu_disc = 0
net.ipv4.route.min_pmtu = 552
net.ipv4.route.mtu_expires = 600
net.ipv4.tcp_mtu_probe_floor = 48
net.ipv4.tcp_mtu_probing = 0
net.ipv4.icmp_echo_ignore_all = 0
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.icmp_errors_use_inbound_ifaddr = 0
net.ipv4.icmp_ignore_bogus_error_responses = 1
net.ipv4.icmp_msgs_burst = 50
net.ipv4.icmp_msgs_per_sec = 1000
net.ipv4.icmp_ratelimit = 1000
net.ipv4.icmp_ratemask = 6168
Note I replaced the ip address with the correct hostname if the output looks a bit wonky.
I'm beat trying to understand why lowering the MTU to any number below a certain threshold results in ping not getting any responses. Inspite of me not messing with the fragmentation bit.
If I use the -M option of do/want/dont for the 1500 byte packet I get a response till hosta MTU stays above 1488 but once it's set to 1487 or below, do/want/dont all get no response.
Following up a bit more on this..
With MTU=1487 tcpdump on that host(=10.50.107.129) shows this incoming 1 IP packet
1 2023-02-15 22:40:24.095129 10.50.107.83 10.50.107.129 IPv4 562 Fragmented IP protocol (proto=ICMP 1, off=1480, ID=1fb9)
That's the only line. Says it's fragmented but shows 562 bytes as the length which is the same length as the one shown below in #3 below.
Next I change MTU=1500 where I see the payload fragmented and acknowledged which is shown below.
2 2023-02-15 22:40:42.093639 10.50.107.83 10.50.107.129 IPv4 1514 Fragmented IP protocol (proto=ICMP 1, off=0, ID=2c62) [Reassembled in #3]
3 2023-02-15 22:40:42.093639 10.50.107.83 10.50.107.129 ICMP 562 Echo (ping) request id=0x1004, seq=1/256, ttl=64 (reply in 5)
4 2023-02-15 22:40:42.093698 10.50.107.129 10.50.107.83 IPv4 1514 Fragmented IP protocol (proto=ICMP 1, off=0, ID=fe1a) [Reassembled in #5]
5 2023-02-15 22:40:42.093717 10.50.107.129 10.50.107.83 ICMP 562 Echo (ping) reply id=0x1004, seq=1/256, ttl=64 (request in 3)
I checked the data in #1 and compared with the data in #4, they are not +/- 3 bytes (ping stuffs hex 00 to ff as the payload). So either the sender didn't send the first fragment or the receiver didn't read the first fragment or something else is going on.
Both servers are built identically as vm's running Ubuntu. So eliminated one side by .
If I run a ping from my laptop through the vpn/firewall/etc/etc, I can ping the host with MTU=1487.
Stumped and would appreciate if someone has any suggestions.
