Score:0

Netplan - Requesting help to properly manage routes in macvlan setup

in flag

I use docker with macvlan where each of my containers have their own L2 connectivity with MAC address and IP. I also need to guarantee that containers reach out the host and vice versa. I can achieve that at this moment, using netplan and a script to delete some routes, but want your help to understand if I can make it using netplan.

My configuration is this one:

Batch script to create macvlan interface (not yet supported on netplan)

#! /bin/bash
#! ip link add macvlan link vlan10 type macvlan mode bridge

vlan10 interface is created with netplan config

01-netcfg.yaml

network:
  version: 2
  renderer: networkd
  ethernets:
    eno1:
      dhcp4: yes
      dhcp6: yes
      addresses:
        - aaaa:bbbb:cccc:1::11/64

  vlans:
    vlan10:
      id: 10
      link: eno1
      dhcp4: no
      dhcp6: no
      addresses:
        - 192.168.10.11/24
        - aaaa:bbbb:cccc:10::11/64

This assures that the VLAN interface created just for docker are create and available.

20-docker.yaml

network:
    version: 2
    renderer: networkd
    ethernets:
        macvlan:
            dhcp4: no
            dhcp6: no
            addresses:
                - 192.168.10.5/24
                - aaaa:bbbb:cccc:10::5/64

I would admit that maybe this is not the most effective way to set up the solution that I require, but it works and in this way I avoid create scripts to be run on reboot using cron.

The routes part

Now that I've created the interfaces, when they are up a set of routes are automatically created, even I do not define gateway4 and gateway6 directives and define netplat to ignore DHCP default routes. But despite that, this routes, for IPv4 and IPv6 are created automatically that avoids reaching containers in vlan10 network due to docker network drivers limitation by design.

IPv4 routes for the new interfaces

192.168.10.0/24 dev vlan10 proto kernel scope link src 192.168.10.11
192.168.10.0/24 dev macvlan proto kernel scope link src 192.168.10.5
192.168.10.0/24 via 192.168.10.5 dev macvlan proto static metric 100
192.168.10.0/24 via 192.168.10.11 dev vlan10 proto static metric 150

IPv6 routes for the new interfaces

aaaa:bbbb:cccc:10::/64 dev vlan10 proto kernel metric 256 pref medium
aaaa:bbbb:cccc:10::/64 dev macvlan proto kernel metric 256 pref medium

For my solution to work, I would need to manually delete (or use a boot bash script) to delete the kernel routes above for device vlan10 in IPv4 and IPv6. By doing that task I can restore the host-container communication.

Is there a way to improve this solution and avoid to create those routes, or a command to ignore that specific route that I do not need on that particular interface to guarantee to use my solution in netplan?

Your expertise and availability is greatly appreciated. Thank you

** 20/10/2021 - Edit after comments and further testing**

After some more reviewing, testing and learning, I've proceeded to the following changes that have solved my issues and permit my host to reach out to containers inside macvlan interface.

01-netcfg.yaml

network:
  version: 2
  renderer: networkd
  ethernets:
    eno1:
      dhcp4: yes
      dhcp6: yes
      addresses:
        - aaaa:bbbb:cccc:dddd:192:168:1:11/64

  vlans:
    vlan10:
      id: 10
      link: eno1
      dhcp4: no
      dhcp6: no
      addresses:
        - 192.168.10.11/32
        - aaaa:bbbb:cccc:10::11/128
      link-local: []

Changed IPv4 and IPv8 subnet masks to /32 and /128 respectively. This have created the proper rules and make my host to restore comms with my docker containers network. Also included `link-local: []´ to avoid this interface to create a link local IP that could interfere.

20-docker.yaml

network:
  version: 2
  renderer: networkd
  ethernets:
    macvlan:
      dhcp4: no
      dhcp6: no
      addresses:
        - 192.168.10.5/24
        - aaaa:bbbb:cccc:10::5/64

With this configuration, I could restore comms between host and containers, for IPv4 and IPv6 and avoid external scripts to remove routes or declare specific routes: and routing-policies: directives in netplan.

Hope this helps others with similar issues. Thanks

Score:0
us flag

These routes are created automatically because you have declared the following in your netplan yaml:

  addresses:
    - 192.168.10.11/24
    - aaaa:bbbb:cccc:10::11/64

If you don't want there to be routes to these networks on the vlan10 interface, you can simply not declare them.

in flag
Thank you for your comment. It makes sense to put /32 and /128 subnet on the `vlan10` interface and force all the routes to go through `macvlan` interface.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.