How to configure 2 x NIC with each having their own gateway?

Granwille

12/2/23, 2:57 PM

I am going to give a brief unrelated explanation first to make sense of my question:

I have 3 Linux boxes with each having 2 x NICs, 1 for the WAN interface and 1 for the LAN interface. Each Linux box has a full public subnet from the ISP configured on the WAN interface and I can successfully reach the "internet" using said interface.

As for the LAN interface on each box, they are configured on the 192.168.50.0/24 network. However, they are isolated from the "internet" and can only ping each other successfully. The ISP provides no NAT/Router functionality for my LAN network.

To solve this first issue, I got a 4th Linux box (with 2 x NIC) and installed OPNsense and now I have a "gateway" (192.168.50.1) for my LAN network. I configured OpenVPN via OPNsense with a tunnel network set as 192.168.10.0/24, and I can successfully connect to it from a remote client and ping my OPNsense server on its private IP address (192.168.50.1). However, I can only ping some clients that have 1 LAN NIC and their gateway setup using the OPNsense server's private IP (192.168.50.1).

For the three initial servers (with 2 x NIC) I still cannot ping them, the only difference is that their LAN NIC does not have the gateway IP (192.168.50.1) specified. However, if I try adding it, as where it works with the other clients, then my server becomes completely unreachable and I need to reconnect via the console to undo the gateway changes. It seems Linux does not like having 2 gateways and I am not exactly sure how to fix this, because I need to reach these three servers too.

Please take note, the issue is not related to OPNsense or OpenVPN because I can reach some clients that have the LAN gateway in place, it's the fact that configuring a Linux box with two gateways causes the server to go down completely.

I have read that in cases like this I need to configure a static route, but to me, that makes no sense. PS all three Linux boxes use Almalinux 8. \

I would appreciate any input in this matter, thanks.

787

1 + 3

nic

gateway

linux-networking

Nikita Kipriyanov

12/3/23, 11:50 AM

Does [this](https://lartc.org/howto/lartc.rpdb.multiple-links.html#AEN267) look like what you need? If yes, do you need a more detailed explanation of how to set it up in your system?

Granwille

12/4/23, 10:15 AM

Thanks Nikia, it does seem this is is the answer, however, Appleoddity gave a good simple explanation and solution to the problem.

Nikita Kipriyanov

12/4/23, 10:59 AM

Fine, although what is written in the answer below is quite different from what is written in LARTC. If you are fine with that, all good.

Score:2

Server

Appleoddity

12/2/23, 9:59 PM

NO system likes multiple active DEFAULT gateways. A default gateway is, by definition, the default path to reach all unknown networks. An unknown network is one which the server does not have a more direct path in the routing table for.

If you have more than one default gateway then both gateways MUST have a path to all the same networks. In this case, they do not and are actually two gateways which sit on two completely separate networks. This causes network packets to egress the wrong network interfaces. This is not the way to implement this.

Keep the default gateway on the WAN interface and remove the gateway all together on the LAN interface. Then setup static routes for the private subnet ranges, which are reachable by the VPN server, to use the next hop of 192.168.50.1. In this case, the network which needs a static route is 192.168.10.0/24. This is done on the three servers with dual interfaces.

The command on the 3 servers looks like this:

ip route add 192.168.10.0/24 via 192.168.50.1

This is temporary and will be lost after a reboot. How to add a permanent static route varies on different flavors of Linux but it should be fairly simple to find with Google.

+ 1

Granwille

12/4/23, 10:15 AM

Hi @Appleoddity, thank you VERY much for your feedback, your solution worked 100% for my 3 boxes. I will find a way to set it permanently up.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: How to configure 2 x NIC with each having their own gateway?

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.