Join Log in
Score:0
sebastien dontneedtoknowthat avatar Server

why port redirection and DNAT does not work with vxlan

jp flag
sebastien dontneedtoknowthat

I need to create VXLAN connections between two computers A (on Debian with IP 192.168.30.1) and B (on archlinux with IP 192.168.30.3) For that, I did:

  • On A:
  sudo ip link add vxlan1 type vxlan id 1 nolearning remote 192.168.30.3 dstport 33333 dev ens4
  sudo ip link set vxlan1 up
  sudo ip addr add 10.0.0.106/24 dev vxlan1
  • On B:
  sudo ip link add vxlan2 type vxlan id 1 nolearning remote 111.111.111.111 dstport 33333 dev ens3
  sudo ip link set vxlan2 up
  sudo ip addr add 10.0.0.107/24 dev vxlan2

Moreover, on PC B I create a DNAT rule:

  sudo iptables -w -t nat -A OUTPUT -s 192.168.30.3 -d 111.111.111.111 -p udp --dport 33333 -j DNAT --to-destination 192.168.30.1:33333

Then I did:

  • On PC A: ping 10.0.0.107. This works as expected with ping replies.
  • On PC B: ping 10.0.0.106. This works as expected with ping replies.
  • On PC A: nc -u -lp 12345. On PC B: nc -u 10.0.0.106 12345. I expected to read data on the netcat application on PC A when sending data with the netcat command from PC B. However I read nothing.

My problem is then at the last point. Why my netcat listener does not receive anything. By using wireshark on PC A I get:

[wireshark dump on PC A1

Some extra information

  • PC A runs with 'Linux debian 4.19.0-16-amd64 #1 SMP Debian 4.19.181-1 (2021-03-19) x86_64 GNU/Linux' (this is the result of uname -a command)

  • PC A IP on ens4 is 192.168.30.1/24

  • PC B runs with 'Linux archlinux 5.13.9-arch1-1 #1 SMP PREEMPT Sun, 08 Aug 2021 11:25:35 +0000 x86_64 GNU/Linux' (this is the result of uname -a command)

  • PC B IP on ens3 is 192.168.30.3/24

  • the two machines are Qemu VMs started with GNS3

  • nat and filter tables of netfilter on PC A is empty:

seb@debian:~$ sudo iptables -t nat -L ; sudo iptables -t filter -L
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
  • On PC B I only have the DNAT rule and a rule to drop ICMP packet of type 'port unreachable' (This is a rule I added for another purpose):
[seb@archlinux vxlan]$ sudo iptables -t nat -L -n ; sudo iptables -t filter -L -n
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
DNAT       udp  --  192.168.30.3         111.111.111.111      udp dpt:33333 to:192.168.30.1:33333

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
DROP       icmp --  0.0.0.0/0            0.0.0.0/0            icmptype 3
71
0 + 6
Michael Hampton avatar
cz flag
Michael Hampton
Why are you using NAT and not normal routing?
0
Reply
sebastien dontneedtoknowthat avatar
jp flag
sebastien dontneedtoknowthat
Why can't I use NAT instead of normal routing? Of course, using NAT here is useless but actually my question is a simplified version of what I really need to do and where I need NAT. Basically, on the complete scenario, one of my computer is behind a NAT (a box of my ISP). Thus the other computer cannot access the first one directly, And I cannot modify the NAT rules on the box. So I am working on this problem. For now, I only succeed to make ping work by playing with timing of my box conntrack. Eventually, I want to use that to perform links aggregation.
0
Reply
sebastien dontneedtoknowthat avatar
jp flag
sebastien dontneedtoknowthat
I realize that my question can also be: how to perform VXLAN connections between two computers A and B with two assumptions. First, every packets sent by A goes through a NAT machine that changes the source information of the packet. Secondly, I don't want to modify the machine that perform the NAT (so I don't want to add static rules on it). However, I am still curious of why using DNAT does not work with VXLAN.
0
Reply
A.B avatar
cl flag
A.B
While I didn't appear to get it initially, I got the issue. On a hunch, this appeared to solve it: `ethtool --offload vxlan2 tx-checksum-ip-generic off` (and turning it back on gets the issue back). No real good explanation here.
1
Reply
sebastien dontneedtoknowthat avatar
jp flag
sebastien dontneedtoknowthat
This indeed works for me too. I don't really know why since I don't see a difference between two packets sent with and without tx-checksum-ip-generic. But for now, this is the best answer I have.
0
Reply
A.B avatar
cl flag
A.B
If you look carefully, Wireshark tells it *didn't* validate checksum for the encapsulated protocol. I can only assume checksum is wrong and if it had validated it, it would have told wrong checksum.
1
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.