I would like to allow OpenVPN users to reach all Docker containers that are part of Overlay network as if they were in the same LAN. The simplest example is 3 containers distributed on 2 hosts, and the hosts are connected through overlay network, and all 3 containers are part of that overlay network.
For that, I have prepared one OpenVPN container (OpenVPN in bridge mode with TAP adapter), and in that container I created a bridge br0 and bridged the eth0 and tap0 interfaces to br0. By this I would say that I bridged my TAP and all OpenVPN clients to the overlay network (I am not sure if this statement is correct).
So far, when my OpenVPN users connects to OpenVPN, and try to ping Dummy Container 1 (which is on the same host with OpenVPN container), the users are able to ping the Dummy Container 1 (and of course are able to ping OpenVPN container also). So far so good!
However, the users are not able to ping the Dummy Container 2, which exists on a different host (but is part of the same overlay with OpenVPN container as mentioned above).
So in short, the VPN users that are connected to OpenVPN container (bridged to Docker overlay network) and try to ping the other containers can only ping the containers on the same host, not on the other host, as if bridging work only on local bridge and not with Overlay network.
It is important also to tell, that from inside OpenVPN container itself I can ping the Dummy Container 2 (and of course Dummy Container 1 as well).
This figure samarize the whole story I told above. Notice the red arrow that represents the connection that I would like to achieve but does not work at the moment (VPN user connects to Host 1 via OpenVPN then try to ping Dummy Container 2).
Some detailed technical information
OpenVPN container
/etc/openvpn # ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: tap0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP group default qlen 1000
link/ether 06:9b:33:89:85:81 brd ff:ff:ff:ff:ff:ff
3: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
link/ether 02:42:0a:08:00:04 brd ff:ff:ff:ff:ff:ff
inet 10.8.0.4/24 scope global br0
valid_lft forever preferred_lft forever
74: eth0@if75: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master br0 state UP group default
link/ether 02:42:0a:08:00:04 brd ff:ff:ff:ff:ff:ff link-netnsid 0
76: eth1@if77: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:ac:12:00:04 brd ff:ff:ff:ff:ff:ff link-netnsid 1
inet 172.18.0.4/16 brd 172.18.255.255 scope global eth1
valid_lft forever preferred_lft forever
/etc/openvpn # ip route
default via 172.18.0.1 dev eth1
10.8.0.0/24 dev br0 proto kernel scope link src 10.8.0.4
172.18.0.0/16 dev eth1 proto kernel scope link src 172.18.0.4
OpenVPN server config:
port 1194
proto udp
dev tap0
ca ca.crt
cert server.crt
key server.key # This file should be kept secret
dh dh.pem
ifconfig-pool-persist ipp.txt
client-to-client
keepalive 10 120
cipher AES-256-GCM
compress lz4-v2
push "compress lz4-v2"
persist-key
persist-tun
status /var/log/openvpn-status.log
verb 3
explicit-exit-notify 1
##### Authentication #####
# A client certificate is not required. The client need to authenticate using username/password only.
verify-client-cert none
# Disables keys renegotiation. The default value is one hour, which disconnects the user every hour and causes the end user to be challenged to reauthorize using a new OTP.
reneg-sec 0
# Loads and uses LDAP plugin config.
plugin /usr/lib/openvpn/plugins/openvpn-auth-ldap.so /etc/openvpn/auth-ldap.conf
# Loads and uses PAM-auth plugin (mainly for 2FA).
plugin /usr/lib/openvpn/plugins/openvpn-plugin-auth-pam.so "openvpn login USERNAME password PASSWORD pin OTP"
Briding is done like this:
openvpn --mktun --dev tap0
ip link set tap0 promisc on up
ip link add name br0 type bridge
ip link set dev br0 up
ip link set dev tap0 master br0
ip link set dev eth0 master br0
ip addr flush dev eth0
ip addr add 10.8.0.4/24 dev br0
OpenVPN is started using the command
VPN_SERVER_HOST_IP_ADDRESS=10.8.0.254
VPN_NETMASK=255.255.255.0
VPN_POOL_START_IP_ADDRESS=10.8.0.50
VPN_POOL_END_IP_ADDRESS=10.8.0.100
openvpn \
--config server.conf \
--server-bridge $VPN_SERVER_HOST_IP_ADDRESS $VPN_NETMASK $VPN_POOL_START_IP_ADDRESS $VPN_POOL_END_IP_ADDRESS
My Dummy Container 2 (very similar to Dummy Container 1 but with different IPs):
/ # ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
35: eth0@if36: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP
link/ether 02:42:0a:08:00:05 brd ff:ff:ff:ff:ff:ff
inet 10.8.0.5/24 brd 10.8.0.255 scope global eth0
valid_lft forever preferred_lft forever
37: eth1@if38: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP
link/ether 02:42:ac:12:00:03 brd ff:ff:ff:ff:ff:ff
inet 172.18.0.3/16 brd 172.18.255.255 scope global eth1
valid_lft forever preferred_lft forever
/ # ip route
default via 172.18.0.1 dev eth1
10.8.0.0/24 dev eth0 scope link src 10.8.0.5
172.18.0.0/16 dev eth1 scope link src 172.18.0.3
