I use docker with macvlan where each of my containers have their own L2 connectivity with MAC address and IP. I also need to guarantee that containers reach out the host and vice versa. I can achieve that at this moment, using netplan and a script to delete some routes, but want your help to understand if I can make it using netplan.
My configuration is this one:
Batch script to create macvlan interface (not yet supported on netplan)
#! /bin/bash
#! ip link add macvlan link vlan10 type macvlan mode bridge
vlan10 interface is created with netplan config
01-netcfg.yaml
network:
version: 2
renderer: networkd
ethernets:
eno1:
dhcp4: yes
dhcp6: yes
addresses:
- aaaa:bbbb:cccc:1::11/64
vlans:
vlan10:
id: 10
link: eno1
dhcp4: no
dhcp6: no
addresses:
- 192.168.10.11/24
- aaaa:bbbb:cccc:10::11/64
This assures that the VLAN interface created just for docker are create and available.
20-docker.yaml
network:
version: 2
renderer: networkd
ethernets:
macvlan:
dhcp4: no
dhcp6: no
addresses:
- 192.168.10.5/24
- aaaa:bbbb:cccc:10::5/64
I would admit that maybe this is not the most effective way to set up the solution that I require, but it works and in this way I avoid create scripts to be run on reboot using cron.
The routes part
Now that I've created the interfaces, when they are up a set of routes are automatically created, even I do not define gateway4 and gateway6 directives and define netplat to ignore DHCP default routes. But despite that, this routes, for IPv4 and IPv6 are created automatically that avoids reaching containers in vlan10 network due to docker network drivers limitation by design.
IPv4 routes for the new interfaces
192.168.10.0/24 dev vlan10 proto kernel scope link src 192.168.10.11
192.168.10.0/24 dev macvlan proto kernel scope link src 192.168.10.5
192.168.10.0/24 via 192.168.10.5 dev macvlan proto static metric 100
192.168.10.0/24 via 192.168.10.11 dev vlan10 proto static metric 150
IPv6 routes for the new interfaces
aaaa:bbbb:cccc:10::/64 dev vlan10 proto kernel metric 256 pref medium
aaaa:bbbb:cccc:10::/64 dev macvlan proto kernel metric 256 pref medium
For my solution to work, I would need to manually delete (or use a boot bash script) to delete the kernel routes above for device vlan10 in IPv4 and IPv6. By doing that task I can restore the host-container communication.
Is there a way to improve this solution and avoid to create those routes, or a command to ignore that specific route that I do not need on that particular interface to guarantee to use my solution in netplan?
Your expertise and availability is greatly appreciated. Thank you
** 20/10/2021 - Edit after comments and further testing**
After some more reviewing, testing and learning, I've proceeded to the following changes that have solved my issues and permit my host to reach out to containers inside macvlan
interface.
01-netcfg.yaml
network:
version: 2
renderer: networkd
ethernets:
eno1:
dhcp4: yes
dhcp6: yes
addresses:
- aaaa:bbbb:cccc:dddd:192:168:1:11/64
vlans:
vlan10:
id: 10
link: eno1
dhcp4: no
dhcp6: no
addresses:
- 192.168.10.11/32
- aaaa:bbbb:cccc:10::11/128
link-local: []
Changed IPv4 and IPv8 subnet masks to /32 and /128 respectively. This have created the proper rules and make my host to restore comms with my docker containers network. Also included `link-local: []´ to avoid this interface to create a link local IP that could interfere.
20-docker.yaml
network:
version: 2
renderer: networkd
ethernets:
macvlan:
dhcp4: no
dhcp6: no
addresses:
- 192.168.10.5/24
- aaaa:bbbb:cccc:10::5/64
With this configuration, I could restore comms between host and containers, for IPv4 and IPv6 and avoid external scripts to remove routes or declare specific routes:
and routing-policies:
directives in netplan.
Hope this helps others with similar issues. Thanks