Score:1

Why is a system sending "ICMP time exceeded in-transit" messages when forwarding packets?

ch flag
WoJ

I have the following setup

| laptop A connected via Wireguard to a server - 192.168.20.3 |
  
| server B with Wireguard (192.168.20.0) and connected to a LAN (192.168.10.2). Forwarding is enabled |
  
| server C connected to LAN (192.168.10.3) | 

The routing table on B is:

# ip r
default via 192.168.10.1 dev enp3s0 proto static
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
172.18.0.0/16 dev br-4f4174b6b888 proto kernel scope link src 172.18.0.1
192.168.10.0/24 dev enp3s0 proto kernel scope link src 192.168.10.2
192.168.20.0/24 dev wg0 proto kernel scope link src 192.168.20.0
192.168.20.3 dev wg0 proto static

I am trying to ping C from A (A pinging B is OK). The ping fails.

When watching the traffic on B, I see

# tcpdump -i any -nn icmp
tcpdump: data link type LINUX_SLL2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
14:10:39.232603 wg0   In  IP 192.168.20.3 > 192.168.10.3: ICMP echo request, id 1, seq 157, length 72
14:10:39.232658 wg0   Out IP 192.168.20.0 > 192.168.20.3: ICMP time exceeded in-transit, length 100
14:10:39.240944 wg0   In  IP 192.168.20.3 > 192.168.10.3: ICMP echo request, id 1, seq 158, length 72
14:10:39.240983 wg0   Out IP 192.168.20.0 > 192.168.20.3: ICMP time exceeded in-transit, length 100
14:10:39.248667 wg0   In  IP 192.168.20.3 > 192.168.10.3: ICMP echo request, id 1, seq 159, length 72

The first packet (in) is the ICMP echo request sent from A to C (via B) The second packet (out) should be B forwarding to C

What does the ICMP time exceeded in-transit mean in this case?


EDIT

I rebooted the server B and now the ping is fine. It is still very weird, though. Below is a single session of a ping from A to C, as seen on B:

PS C:\Users\X> tracert -d 192.168.10.3

Tracing route to 192.168.10.3 over a maximum of 30 hops

  1     6 ms     6 ms     8 ms  192.168.20.0
  2     6 ms     6 ms     6 ms  192.168.10.3



root@srv ~# tcpdump -i any -nn icmp
tcpdump: data link type LINUX_SLL2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
15:01:39.235308 wg0   In  IP 192.168.20.3 > 192.168.10.3: ICMP echo request, id 1, seq 180, length 72
15:01:39.235355 wg0   Out IP 192.168.20.0 > 192.168.20.3: ICMP time exceeded in-transit, length 100
15:01:39.243813 wg0   In  IP 192.168.20.3 > 192.168.10.3: ICMP echo request, id 1, seq 181, length 72
15:01:39.243854 wg0   Out IP 192.168.20.0 > 192.168.20.3: ICMP time exceeded in-transit, length 100
15:01:39.251044 wg0   In  IP 192.168.20.3 > 192.168.10.3: ICMP echo request, id 1, seq 182, length 72
15:01:39.251075 wg0   Out IP 192.168.20.0 > 192.168.20.3: ICMP time exceeded in-transit, length 100
15:01:39.783426 wg0   Out IP 192.168.20.0 > 192.168.20.3: ICMP host 192.168.10.188 unreachable, length 82
15:01:39.783455 wg0   Out IP 192.168.20.0 > 192.168.20.3: ICMP host 192.168.10.188 unreachable, length 82
15:01:40.262055 wg0   In  IP 192.168.20.3 > 192.168.10.3: ICMP echo request, id 1, seq 183, length 72
15:01:40.262083 enp3s0 Out IP 192.168.20.3 > 192.168.10.3: ICMP echo request, id 1, seq 183, length 72
15:01:40.262509 enp3s0 In  IP 192.168.10.3 > 192.168.20.3: ICMP echo reply, id 1, seq 183, length 72
15:01:40.262530 wg0   Out IP 192.168.10.3 > 192.168.20.3: ICMP echo reply, id 1, seq 183, length 72
15:01:40.270041 wg0   In  IP 192.168.20.3 > 192.168.10.3: ICMP echo request, id 1, seq 184, length 72
15:01:40.270068 enp3s0 Out IP 192.168.20.3 > 192.168.10.3: ICMP echo request, id 1, seq 184, length 72
15:01:40.270423 enp3s0 In  IP 192.168.10.3 > 192.168.20.3: ICMP echo reply, id 1, seq 184, length 72
15:01:40.270440 wg0   Out IP 192.168.10.3 > 192.168.20.3: ICMP echo reply, id 1, seq 184, length 72
15:01:40.278474 wg0   In  IP 192.168.20.3 > 192.168.10.3: ICMP echo request, id 1, seq 185, length 72
15:01:40.278493 enp3s0 Out IP 192.168.20.3 > 192.168.10.3: ICMP echo request, id 1, seq 185, length 72
15:01:40.278829 enp3s0 In  IP 192.168.10.3 > 192.168.20.3: ICMP echo reply, id 1, seq 185, length 72
15:01:40.278842 wg0   Out IP 192.168.10.3 > 192.168.20.3: ICMP echo reply, id 1, seq 185, length 72

It takes a few packets with ICMP time exceeded in-transit to finally have correct traffic go through.


Note: this is the "to" route of the packet, the "reply" one will go through C's default gateway that knows the route for 192.168.20.0/24

Anton Danilov avatar
cn flag
Check the firewall. Paste the output of `iptables-save -c`.
Paul avatar
cn flag
Off-topic, but what is the character in the second line of the first code block? FF can't display it.
ch flag
WoJ
@AntonDanilov: there is no firewall (except on the ISP box where the port is forwarded to the Wireguard server). One the tunnel is established there is no firewall.
ch flag
WoJ
@Paul this is a unicode Downwards Arrow (https://unicode-table.com/en/1F817/). Maybe in the comments section you will see it better:
Anton Danilov avatar
cn flag
Check the `ip route get <C-ip> from <A-ip> iif wg0` and vise versa on B host. Add `-v` option to `tcpdump` command to display the original TTL value in ip packets.
Score:1
jp flag

ICMP time exceeded in-transit means that the TTL of the packet reached zero, most probably due to a routing loop.

ch flag
WoJ
Yes, I know the technical reason for the message - it is just that I do not see here any loop in routing (`traceroute` stops at the first hop for instance)
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.