I have a Centos box with 2 interfaces that have routes to the internet. One 192.168.1.254, the other 192.168.133.11. I can route through 133.11 perfectly OK when it is the only route specified. When I add the other route to the table, the first stops working.
[root@sinister2 yum.repos.d]# ip route
default via 192.168.133.11 dev ens2u3
192.168.1.0/24 dev eno1 proto kernel scope link src 192.168.1.70 metric 100
192.168.133.0/24 dev ens2u3 proto kernel scope link src 192.168.133.108
[root@sinister2 yum.repos.d]# curl ifconfig.me
92.40.187.117
[root@sinister2 yum.repos.d]# route del default
[root@sinister2 yum.repos.d]# ip route add default scope global nexthop via 192.168.133.11 dev ens2u3 weight 1 nexthop via 192.168.1.254 dev eno1 weight 1
[root@sinister2 yum.repos.d]# curl ifconfig.me
86.133.6.21
[root@sinister2 yum.repos.d]# ip route
default
nexthop via 192.168.133.11 dev ens2u3 weight 1
nexthop via 192.168.1.254 dev eno1 weight 1
192.168.1.0/24 dev eno1 proto kernel scope link src 192.168.1.70 metric 100
192.168.133.0/24 dev ens2u3 proto kernel scope link src 192.168.133.108
[root@sinister2 yum.repos.d]# curl https://developer.download.nvidia.com/
curl: (7) Failed to connect to developer.download.nvidia.com port 443: Connection timed out
[root@sinister2 yum.repos.d]# traceroute developer.download.nvidia.com
traceroute to developer.download.nvidia.com (152.199.20.126), 30 hops max, 60 byte packets
1 * * *
...manually clipped all the stars...
30 * * *
[root@sinister2 yum.repos.d]# route del default
[root@sinister2 yum.repos.d]# route add default gw 192.168.133.11
[root@sinister2 yum.repos.d]# traceroute developer.download.nvidia.com
traceroute to developer.download.nvidia.com (152.199.20.126), 30 hops max, 60 byte packets
1 _gateway (192.168.133.11) 2.525 ms 2.667 ms 2.942 ms
2 * * *
3 172.25.83.33 (172.25.83.33) 301.910 ms 307.052 ms 306.962 ms
4 * * *
5 * * *
6 172.25.88.89 (172.25.88.89) 305.709 ms * 172.25.88.93 (172.25.88.93) 292.299 ms
7 * * *
8 172.25.67.146 (172.25.67.146) 48.475 ms 172.25.67.158 (172.25.67.158) 53.182 ms 172.25.67.146 (172.25.67.146) 53.096 ms
9 * * *
10 185.153.238.161 (185.153.238.161) 46.396 ms 57.785 ms 57.785 ms
11 ae-65.core1.lhc.edgecastcdn.net (152.195.96.139) 29.347 ms 39.733 ms 51.525 ms
12 152.199.20.126 (152.199.20.126) 39.295 ms 39.182 ms 39.041 ms
13 152.199.20.126 (152.199.20.126) 39.250 ms 39.251 ms 39.015 ms
That nvidia site routes through ens2u3, I can see the traffic on the interface :
tcpdump -i ens2u3
16:26:37.528360 ARP, Request who-has 192.168.133.11 tell sinister2, length 28
16:26:37.530385 ARP, Reply 192.168.133.11 is-at 76:e6:8d:8e:99:a8 (oui Unknown), length 28
16:26:39.256368 IP sinister2.42630 > 152.199.20.126.https: Flags [S], seq 1095059191, win 29200, options [mss 1460,sackOK,TS val 1024650559 ecr 0,nop,wscale 7], length 0
16:26:47.768368 IP sinister2.42630 > 152.199.20.126.https: Flags [S], seq 1095059191, win 29200, options [mss 1460,sackOK,TS val 1024659071 ecr 0,nop,wscale 7], length 0
etc...
None of my syns ever got an ack. Earlier, when I was routing everything through ens2u3, it did :
16:23:18.233472 IP sinister2.55236 > 152.199.20.126.https: Flags [S], seq 1471928043, win 29200, options [mss 1460,sackOK,TS val 466587424 ecr 0,nop,wscale 7], length 0
16:23:18.276348 IP 152.199.20.126.https > sinister2.55236: Flags [S.], seq 2554681057, ack 1471928044, win 14600, options [mss 1220,nop,wscale 12,sackOK,TS val 371740 3495 ecr 466587424], length 0
16:23:18.276428 IP sinister2.55236 > 152.199.20.126.https: Flags [.], ack 1, win 229, options [nop,nop,TS val 466587467 ecr 3717403495], length 0
Traceroute just shows stars for any address over ens2u3 and a connection to anything on eno1.
Anyone got any good ideas? Spot anything I've done wrong?
Do I need to do some sort of clearing of existing routes when changing them? In the first example, I can get to ifconfig.me via the new route, but not use the old route to nvidia, which makes me think it isn't that simple...
