Score:0

Setting up secondary interface with netplan using DHCP - both interfaces in same subnet

vn flag

I am trying to configure two Network interfaces on an EC2 instance using Netplan. These interfaces have the same subnet. When the cloud-init configures it, the netplan configuration looks as follows:

network:
ethernets:
    eth0:
        dhcp4: true
        dhcp4-overrides:
            route-metric: 100
        dhcp6: false
        match:
            macaddress: xx:xx:xx:xx:xx:xx
        set-name: eth0
    eth1:
        dhcp4: true
        dhcp4-overrides:
            route-metric: 200
        dhcp6: false
        match:
            macaddress: yy:yy:yy:yy:yy:yy
        set-name: eth1
version: 2

The routing for these interfaces are setup in the same OS route table. The rules are as follows:

# ip route show 
default via 172.31.0.1 dev eth0 proto dhcp src 172.31.7.250 metric 100
default via 172.31.0.1 dev eth1 proto dhcp src 172.31.5.137 metric 200
172.31.0.0/20 dev eth0 proto kernel scope link src 172.31.7.250 metric 100
172.31.0.0/20 dev eth1 proto kernel scope link src 172.31.5.137 metric 200
172.31.0.1 dev eth0 proto dhcp scope link src 172.31.7.250 metric 100
172.31.0.1 dev eth1 proto dhcp scope link src 172.31.5.137 metric 200
172.31.0.2 dev eth0 proto dhcp scope link src 172.31.7.250 metric 100
172.31.0.2 dev eth1 proto dhcp scope link src 172.31.5.137 metric 200

Now the problem comes when the traffic comes from secondary interface such that the services on the EC2 are not accessible from secondary interface if made outside from the subnet.

Taking the packet capture on the EC2, it is determined that even though the request comes in from eth1, it goes out via eth0.

# tcpdump -nni any host ${HOME_ADDRESS}
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
02:01:18.036735 eth1  In  IP ${HOME_ADDRESS} > 172.31.5.137: ICMP echo request, id 36766, seq 5, length 40
02:01:18.036765 eth0  Out IP 172.31.5.137 > ${HOME_ADDRESS}: ICMP echo reply, id 36766, seq 5, length 40

Checking few links such as: https://wbhegedus.me/avoiding-asymmetric-routing/ It looks like even though the source is setup for the default route, the default route with metric 100 is prioritized as a result the packet comes out of eth0 rather than eth1.

For now, the solution that I have found is to setup static routes for the eth1, specifying a different route table with following netplan configuration:

network:
renderer: networkd
ethernets:
    eth0:
        dhcp4: true
        dhcp4-overrides:
            route-metric: 100
        dhcp6: false
        match:
            macaddress: xx:xx:xx:xx:xx:xx
        set-name: eth0
    eth1:
        dhcp4: false
        addresses:
            - 172.31.5.137/20
        routes:
        -   to: 172.31.0.0/20
            via: 172.31.5.137
            table: 101
        -   to: 0.0.0.0/0
            via: 172.31.0.1 # Default gateway
            table: 101
        routing-policy:
        -   from: 172.31.5.137
            table: 101
        match:
            macaddress: yy:yy:yy:yy:yy:yy
        set-name: eth1
version: 2

This works and I can verify that the ping goes out via same interface it comes from.

# tcpdump -nni any host ${HOME_ADDRESS}
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
02:01:18.036735 eth1  In  IP ${HOME_ADDRESS} > 172.31.5.137: ICMP echo request, id 36766, seq 5, length 40
02:01:18.036765 eth1  Out IP 172.31.5.137 > ${HOME_ADDRESS}: ICMP echo reply, id 36766, seq 5, length 40

That being said, this configuration is not portable as everytime I have to add a new interface, I have to add static routes. So I was looking for a way to mix different route table approach with dhcp to make it more dynamic.

Is there are way to setup secondary network interfaces with netplan using DHCP with routes being created in a different route table ? I checked network dispatcher but did not find any examples on how to do this.

Any help would be appreciated. OS: ubuntu 22.04

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.