Score:0

Server

Routing between two network cards

Adi

8/29/23, 8:42 AM

I'm trying to route traffic between different networks and have followed a guide I found here: https://devconnected.com/how-to-add-route-on-linux/

Here is a diagram which I hope adequately depicts the arrangment i'm working with:

Windows 10              Ubuntu                            Linux
172.31.0.X <----------> 172.31.0.33 (eno1)
                        10.0.40.1 (enp2s0f0) <----------> 10.0.40.10

I have a persistant route set on the Windows PC to route any traffic for 10.0.40.0/24 via 172.31.0.33.

Route Print output

The Ubuntu machine is set up to route traffic destined for 10.0.40.0/24 via 10.0.40.1.

ip r output

Pinging 10.0.40.10 from the Ubuntu machine works as expected.

If I ping 10.0.40.10 from the Windows PC I can see the ICMP messages arrive at the 172.31.0.33 interface on the Ubuntu machine using tcpdump. I don't see any traffic on the 10.0.40.1 interface on that machine. It appears that the Ubuntu machine is not routing the traffic as I would expect. Can anyone shed any light on what I have missed?

Adding output of:

iptables -S

for Ubuntu machine:

sudo iptables -S
# Warning: iptables-legacy tables present, use iptables-legacy to see them
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-A FORWARD -i eno1 -j ACCEPT
-A FORWARD -i enp2s0f0 -j ACCEPT

adi@LabBuildServer:~$ sudo iptables-legacy -S
[sudo] password for adi:
-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-ISOLATION-STAGE-1
-N DOCKER-ISOLATION-STAGE-2
-N DOCKER-USER
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -o br-e925d11be2da -m conntrack --ctstate RELATED,ESTABLISHED -j ACCE                         PT
-A FORWARD -o br-e925d11be2da -j DOCKER
-A FORWARD -i br-e925d11be2da ! -o br-e925d11be2da -j ACCEPT
-A FORWARD -i br-e925d11be2da -o br-e925d11be2da -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i br-e925d11be2da ! -o br-e925d11be2da -j DOCKER-IS                         OLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-e925d11be2da -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN

Output of:

ip route

on Linux host:

ip route
default via 10.0.40.1 dev br-POE  proto static
10.0.40.0/24 dev br-POE  proto kernel  scope link  src 10.0.40.10

Ubuntu machine:

adi@LabBuildServer:~$ sudo iptables -t nat -L
[sudo] password for adi:
# Warning: iptables-legacy tables present, use iptables-legacy to see them
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  all  --  anywhere             anywhere
MASQUERADE  all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

0 + 0

linux

routing

Lacek

8/29/23, 9:05 AM

Is `/proc/sys/net/ipv4/ip_forward` set to `1`? I know, basic stuff, but I lost count how many times I made my life harder by forgetting it.

Adi

8/29/23, 11:06 AM

Hi @Lacek. Yes it is set to 1. Always worth checking! Thanks.

4snok

8/29/23, 2:04 PM

do you have any rules in the iptables forward chain on Ubuntu machine? Does Linux machine know 172.31.0.x network?

Adi

8/29/23, 2:53 PM

Hi @4snok i've not added any iptables rules. Following the guide I linked there is no mention of that to get ping working. Is that something I need?

A.B

8/29/23, 2:54 PM

Why don't you put in the question here at serverfault what you did (in text not screenshots)? We don't really want to have to navigate elsewhere to a tutorial to try and see what you actually did or didn't.

4snok

8/29/23, 3:32 PM

please add to the questions output of `iptables -S` from Ubuntu host and `ip route` from Linux host

Adi

9/3/23, 9:38 AM

Hi @4snok i've added the outputs as requested. Does that tell you anything?

4snok

9/3/23, 11:33 AM

not really :(, could you check if there are any fw rules with `iptables-legacy -S` as iptables suggested?

4snok

9/3/23, 11:43 AM

try to ping linux from Ubuntu 172.31. interface, `ping -S 172.31.0.33 10.0.40.10` make sure it's working. Also I found the routing table on windows machine quite confusing e.g. in your persistent route you specify gw `172.31.0.33`, but according to the routing table 172.31.0.33 is reachable via 10.255.255.0, there is no on-link rule

djdomi

9/3/23, 5:10 PM

Basically, is nat used?

Adi

9/4/23, 7:42 AM

Did you mean `ping -S` or `ping -I`? `ping -I 172.31.0.33 10.0.40.10` works as expected.

Score:1

Server

Martin

9/4/23, 9:34 AM

The solution is right there:

-P FORWARD DROP

In your iptables-legacy ruleset, the default policy for forwarding pakets is set to DROP, and there is no rule in that ruleset which allows forwarding pakets from eno1 to enp2s0f0, only forwarding from/to a bridge interface...

It is always a very bad idea to mix different iptables, you should decide yourself if you want to use iptables or iptables-legacy - each network paket will go through both rulesets, causing quite a lot of confusion.

update:
My answer should not mean that you have to install the default policy as accept, I was just pointing to the reason. Of course you can add rules to allow forwarding traffic to these specific IPs only, for example like this:

-A FORWARD -i eno1 -s 172.31.0.0/24 -o enp2s0f0 -d 10.0.40.0/24 -j ACCEPT
-A FORWARD -i enp2s0f0 -s 10.0.40.0/24 -o eno1 -d 172.31.0.0/24 -j ACCEPT

0 + 0

Adi

9/4/23, 1:52 PM

Thanks. Yes it looks obvious when you know where to look! Rather than just changing that to `-P FORWARD ACCEPT' would it be possible to leave that as DROP and then add a specific rule just to forward traffic from 172.31.0.0/24 to 10.0.40.0/24?

Martin

9/4/23, 2:53 PM

Of course it is possible! I added an example, how the rules could look like, to the answer...

A.B

9/4/23, 6:50 PM

Don't forget this is all due to Docker interacting with network settings: https://docs.docker.com/network/iptables/ . When troubleshooting something that doesn't depend on Docker on a system where Docker is running, the first thing to do should be to disable Docker and reboot, then try again. Stopping Docker (and not rebooting) won't remove everything (eg: the loaded module `br_netfilter` can make any other setup that would use its own bridge to fail without apparent reason).

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Routing between two network cards

TH: การกำหนดเส้นทางระหว่างการ์ดเครือข่ายสองการ์ด

RO: Rutare între două plăci de rețea

RU: Маршрутизация между двумя сетевыми картами

VI: Định tuyến giữa hai card mạng

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.