Ping across IPSec tunnel sends both ICMP and ESP packets

I've been using Strongswan to setup an IPSec tunnel between two units. The tunnels SAs get setup without any issues and traffic can pass across the connection.

Whenever I ping across the tunnel, the ping request is sent as BOTH an ESP and ICMP packet. The ping response is always just an ESP packet. Looking at these packets with Wireshark, it seems like the ping request sends two packets, one encapsulated and one not. The response is just an encapsulated ICMP echo response. So every ping results in 3 packets exchanged rather than 2.

The units can communicate without an IPSec tunnel but the same issue happens when the IPSec connection is over a GRE tunnel.

How does one diagnose/fix this issue?

Example tcpdump

tcpdump -vv host yyy.yyy.yyy.yyy
tcpdump: listening on enp4s0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:18:33.302502 IP (tos 0x0, ttl 64, id 11718, offset 0, flags [DF], proto ESP (50), length 156)
    unit1 > unit2: ESP(spi=0xc3e93e85,seq=0x1), length 136
10:18:33.302762 IP (tos 0x0, ttl 64, id 5550, offset 0, flags [DF], proto ICMP (1), length 84)
    unit1 > unit2: ICMP echo request, id 30684, seq 1, length 64
10:18:33.303033 IP (tos 0x0, ttl 64, id 1623, offset 0, flags [none], proto ESP (50), length 156)
    unit2 > unit1: ESP(spi=0xcc947f6d,seq=0x1), length 136

IPSec status

ipsec status
Security Associations (1 up, 0 connecting):
    unit-to-unit[2]: ESTABLISHED 11 minutes ago, yyy.yyy.yyy.yyy[yyy.yyy.yyy.yyy]...zzz.zzz.zzz.zzz[zzz.zzz.zzz.zzz]
    unit-to-unit{1}:  INSTALLED, TUNNEL, reqid 1, ESP SPIs: cc947f6d_i c3e93e85_o
    unit-to-unit{1}:   yyy.yyy.yyy.yyy/32 === zzz.zzz.zzz.zzz/32

/etc/ipsec.conf

config setup
    #strictcrlpolicy=yes
    #uniqueids = no
    #nat_traversal=yes

conn unit-to-unit
    authby=secret
    auto=start
    keyexchange=ikev2
    ike=aes256-sha2_256-modp1024!
    left=yyy.yyy.yyy.yyy
    right=zzz.zzz.zzz.zzz
    type=tunnel
    esp=aes256-sha2_256

I believe that is an artefact of how IPsec and packet capture work on Linux, the packet is simply captured twice on the sender side.

If you check on the other host, you'll see that the request comes in as a single ESP packet, and the reply is captured twice, once as ICMP, once as ESP.

On an intermediate system, you'd only be able to see the ESP packets.