Join Log in
Score:1
avatar Ubuntu

Ubuntu Server Netplan - conflicting default route declarations for IPv4 - 4 different /24s on each port

ru flag
lion

I have a server with 2 network cards of 4 ports, but only one of them is used. Each port is connected to a L2 switch and configured for a different VLAN with 1 usable IP from a different /24 subnet - each with different default gateway. Only 1 of the ports works properly - eno4. The rest could be pinged, but with losses every other time. I get this error when applying netplan:

Problem encountered while validating default route consistency.Please set up multiple routing tables and use routing-policy instead. Error: Conflicting default route declarations for IPv4 (table: main, metric: default), first declared in eno2 but also in eno4

netplan itself:

network:
  ethernets:
    eno1:
      addresses: ['aa.bb.cc.dd/24', 'a:b:c:x:e:f:g:1/64']
      gateway4: aa.bb.cc.1
      gateway6: a:b:c:x:e:f:g:0001
      nameservers:
          addresses: [1.1.1.1, 1.0.0.1, 8.8.8.8, 8.8.4.4, 2606:4700:4700::1111, 2606:4700:4700::1001, 2001:4860:4860::8888, 2001:4860:4860::8844]
      accept-ra: no
    eno2:
      addresses: ['ee.ff.gg.hh/24', 'a:b:c:y:e:f:g:2/64']
      gateway4: ee.ff.gg.1
      gateway6: a:b:c:y:e:f:g:0001
      nameservers:
          addresses: [1.1.1.1, 1.0.0.1, 8.8.8.8, 8.8.4.4, 2606:4700:4700::1111, 2606:4700:4700::1001, 2001:4860:4860::8888, 2001:4860:4860::8844]
      accept-ra: no
    eno3:
      addresses: ['ii.jj.kk.ll/24', 'a:b:c:z:e:f:g:3/64']
      gateway4: ii.jj.kk.1
      gateway6: a:b:c:z:e:f:g:0001
      nameservers:
          addresses: [1.1.1.1, 1.0.0.1, 8.8.8.8, 8.8.4.4, 2606:4700:4700::1111, 2606:4700:4700::1001, 2001:4860:4860::8888, 2001:4860:4860::8844]
      accept-ra: no
    eno4:
      addresses: ['mm.nn.oo.pp/24', 'a:b:c:v:e:f:g:4/64']
      gateway4: mm.nn.oo.1
      gateway6: a:b:c:v:e:f:g:0001
      nameservers:
          addresses: [1.1.1.1, 1.0.0.1, 8.8.8.8, 8.8.4.4, 2606:4700:4700::1111, 2606:4700:4700::1001, 2001:4860:4860::8888, 2001:4860:4860::8844]
      accept-ra: no
#    ens1f0:
#      dhcp4: true
#    ens1f1:
#      dhcp4: true
#    ens1f2:
#      dhcp4: true
#    ens1f3:
#      dhcp4: true
  version: 2

What am I missing here and how to properly route?

1497
0 + 0
Score:1
user10489 avatar Ubuntu
in flag
user10489

There typically can be only one functional default route. The default route is where packets go when there is no obvious better route. If you have multiple default routes, they might be treated round robin style, which would explain why you have packet loss -- some packets are going to different places.

There are multiple possible solutions to this:

  • Use a routing daemon to manage dynamic routes, outside of netplan
  • Assign static routes for specific networks with one default route for everything else.
  • Set up routing tables and priorities so that the selected "default route" depends on the incoming host side IP address, so that responses to packets coming in on a specific ip go out the same interface. Similarly, you would need to prioritize it so that wildcarded outgoing connections select a primary IP for the source address.

It may be necessary to do more than one of these things to make it work.

0 + 0
ru flag
lion
OK. Thanks! The last one worked. ip rule add from IP1 table 1 and then ip route add default via GATE1 table 1 Same for the rest. Added it to startup script, so it is applied across reboots.
0
Reply
user10489 avatar
in flag
user10489
There should be a way to do that inside netplan, but I'm not sure.
0
Reply
ru flag
lion
Same here, so did what I said and seems to have worked.
0
Reply
ru flag
lion
So this worked for the outside world, but not for docker containers. How to write static routing for each separate container?
0
Reply
user10489 avatar
in flag
user10489
not enough context to answer, maybe worth a separate question. Ultimately, every packet needs a consistent destination, and every reply must go out on the same ip the corresponding incoming packet came to.
0
Reply
ru flag
lion
Was a firewall thing. I solved it. :)
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.