Score:0

Server

Isolating Docker Bridge Network with VLANs

Jameson_uk

7/10/24, 12:38 PM

My network is isolated into a few different VLANs but I am struggling a little with the best way to achieve isolation of some docker containers running inside a custom bridge network. For the sake of simplicity lets say the network looks like this and there are two VLANs (Trusted traffic is untagged and non-trusted traffic is on VLAN 10). Untagged traffic is on a 192.168.x.0/24 subnet and VLAN 10 is on a 10.x.x.0/24 subnet.

I am running docker on Unraid and have set it up to allow VLANs and it gets an IP on both the untagged and VLAN 10 subnets. Unraid is configured so that is just docker that is accessible on VLAN 10 and with some containers running in a custom bridge network (10.99.99.0/24) everything works one way. When mapping ports to containers I use <VLAN 10 IP>:Port so that the port is only accessible via the VLAN 10 IP. This allows me to have some containers running inside the docker network that are purely internal to that network and not accessible outside.

The issue I have is that the containers themselves can communicate back out to the untagged network. I guess this makes sense as the host has a route to the untagged network and the docker traffic isn't tagged with a VLAN ID.

I have tried creating a iptables rule to drop traffic between the docker 10.99.99.0/24 interface and br0

iptables -A FORWARD -i br-<MAC> -o br0 -j DROP

but that doesn't seem to work and traffic can still get through.

How can I add outbound isolation to this setup? (ie. I only want to allow traffic from this docker network out through the br.10 interface)

I don't want to just use a MACVLAN network and put the devices on VLAN 10 as their are ports on containers I don't want to be accessible to other devices and there are some containers that I simply don't want to be seen at all on that subnet.

Ideally it would be nice to do all of this at the host level but the only other option I can see is to create a MACVLAN network on a new VLAN and apply lots of firewall rules on the router (which is a bit of a pain to maintain as I will then need to fix IPs of the containers etc.)

1 + 4

docker

docker-networking

larsks

7/10/24, 2:13 PM

"I have tried creating a iptables rule to drop traffic..." What did the rule look like?

Jameson_uk

7/10/24, 2:20 PM

Added above but was the interface that that shows up with the 10.99.99.1/24 subnet and has the MAC generated for it as part of the interface name `iptables -A FORWARD -i br-<MAC> -o br0 -j DROP`

larsks

7/10/24, 2:25 PM

Please don't anonymize "private" addresses. There's no reason to write `10.x.x.0`; you can just use the actual address. This makes it much easier to write about and reason about things.

Jameson_uk

7/10/24, 2:42 PM

It isn't anonymised... I changed them all recently and I can't actually remember what they actually are :$ I will change them when I get home check to check. Either way there is a 192. untagged, 10. vlan 10 and I know the docker one is 10.99.99.1/24

Score:1

Server

larsks

7/10/24, 2:26 PM

An iptables rule should be sufficient. It would need to be at the top of your FORWARD chain; something like:

iptables -A FORWARD -s 10.99.99.0/24 -d 192.168.x.0/24 -j DROP

It needs to be at the top of the chain because otherwise it becomes a no-op -- there are rules added by Docker that will explicitly ACCEPT the traffic.

You could instead add the rule to the DOCKER-USER chain; this is a chain that Docker arranges to be called before any Docker-managed rules. The FORWARD chain on my local system looks like:

-P FORWARD ACCEPT
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
.
.
.

Alternately, you could add a policy routing rule so that traffic from your containers wouldn't have a route to the untagged network. Your default routing policy looks like this:

# ip rule show
0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup default

Where the main table is what you see when you run ip route. You can add a rule that uses an alternate lookup for traffic originating from your containers:

ip rule add priority 1000 from 10.99.99.0/24 lookup 1000

That should result in:

# ip rule show
0:      from all lookup local
1000:   from 10.99.99.0/24 lookup 1000
32766:  from all lookup main
32767:  from all lookup default

And then add routing entries to table 1000:

ip route add table 1000 default via 10.x.x.1

Now connections from the containers will only have a default route, and would only be able to access the untagged network if the router at 10.x.x.1 provided an appropriate route.

+ 1

Jameson_uk

7/10/24, 3:42 PM

It was indeed the top bit that was missing... `iptables -I DOCKER-USER 1 -s 10.99.99.0/24 -d 192.168.40.0/24 -j DROP` drops all traffic. Turns out I need to teak this slightly but this is the correct answer so thanks

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Isolating Docker Bridge Network with VLANs

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.