How "ip route get" decides outgoing interface

Manny

11/12/23, 7:02 AM

Wondering what decides the outgoing interface. Here is my setup

Eno1 - 192.168.3 0.2/24, Gateway - 192.168.30.1
Eno2 - 192.168.50.2/24,  Gateway - 192.168.50.1

My routing table looks like this -

[root ~]# ip route
default
    nexthop via 192.168.30.2 dev eno1 weight 1
    nexthop via 192.168.50.2 dev eno2 weight 1
192.168.30.0/24 dev eno1 proto kernel scope link src 192.168.30.2 metric 100
192.168.50.0/24 dev eno2 proto kernel scope link src 192.168.50.2 metric 101

And

Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
default         hostname        0.0.0.0         UG        0 0          0 eno1
192.168.30.0    0.0.0.0         255.255.255.0   U         0 0          0 eno1
192.168.50.0    0.0.0.0         255.255.255.0   U         0 0          0 eno2

After this config, when I try to ask linux kernel which interface it will use for a random destination...it plays a ping pong game between eno1 and eno2

[root@ ~]# ip route get 1.1.1.9
1.1.1.9 via 192.168.50.2 dev eno2 src 192.168.50.2
    cache
[root@ ~]# ip route get 1.1.1.10
1.1.1.10 via 192.168.30.2 dev eno1 src 192.168.30.2
    cache
[root@ ~]# ip route get 1.1.1.12
1.1.1.12 via 192.168.50.2 dev eno2 src 192.168.50.2
    cache
[root@-2 ~]# ip route get 1.1.1.13
1.1.1.13 via 192.168.30.2 dev eno1 src 192.168.30.2
    cache
[root@-2 ~]# ip route get 1.1.1.14
1.1.1.14 via 192.168.30.2 dev eno1 src 192.168.30.2

it feels like the outgoing interface is selected at random Can you please help understand what decide the outgoing interface?

766

1 + 0

networking

linux

linux-networking

iproute2

Score:0

Server

casual1st

11/12/23, 8:21 AM

This is called Equal-Cost Multipath (ECMP).

Linux is designed to distribute flows of packets using multiple paths. It will not happen for individual packets. The network stack calculates the hash over a subset of packet header fields. The calculated hash will be used for the next hop selection.

Note that balancing will not be perfect and there may be significant problems in the deployment in practice, as it is route-based, and routes are cached. This means often-used IPs will always use the same route.

+ 1

Manny

11/14/23, 11:34 PM

Thank you very much, this makes total sense. Can you help me understand if there is a way to failover? In other words, if a destination is not reachable from NIC1 and it receives a ICMP message host unreachable...how do I failover and make it use NIC2 and then give up instead of a single try/NIC?

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: How "ip route get" decides outgoing interface

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.