Join Log in
Score:0
cpl avatar Server

Container bridge networking with subnet on Hetzner and CNI

us flag
cpl

I am trying to create a private network of VMs and containers on Hetzner Cloud. I've tested this setup on my local home network and all works fine.

The plan is to have a private network for the VMs (10.0.0.0/8 just for testing). For my test I am using 10.10.0.1 and 10.10.0.2 as VMs. And each VM, will have a CNI bridge network, (eg: 172.20.0.0/16, for container). I want to make the bridge networks accesible by any VM on the 10.0.0.0/8 network, with static routes.

On Hetzner, I've configured a static route 172.20.0.0/16 to 10.10.0.1. On 10.10.0.1 I have a CNI bridge network for Podman, configured on the same range 172.20.0.0/16.

Any container that gets placed on that network, has no trouble pinging or reaching out to: local, other containers, host or internet, and the host (10.10.0.1) has no trouble reaching the containers (172.20.0.X).

The problem is when I want to ping the container from 10.10.0.2. I've monitored traffic with tcpdump and iftop, and the Hetzner route seems to work just fine, as connections reach the VM on 10.10.0.1 (ens10). Which makes me wonder if it's a routing issue between the ens10 and podman-vlan bridge interfaces?

Here are the routes from 10.10.0.1

default via 172.31.1.1 dev eth0 proto dhcp src X.Y.Z.W metric 100 
10.0.0.0/8 via 10.0.0.1 dev ens10 
10.0.0.1 dev ens10 scope link 
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 
172.20.0.0/16 dev podman-vlan proto kernel scope link src 172.20.0.1 
172.31.1.1 dev eth0 proto dhcp scope link src X.Y.Z.W metric 100

On the 10.10.0.2 VM, I've only done a ip r add 172.20.0.0/16 via 10.0.0.1 (which seems to work as).

My expectation was to get 10.10.0.2 -> 10.0.0.1 -> 10.10.0.1 -> 172.20.0.1 -> 172.20.0.X. Instead everything seems to get lost at 10.10.0.1, including if I try ping -I ens10 172.20.0.X

This is the CNI config:

{
  "cniVersion": "0.4.0",
  "name": "podman",
  "plugins": [
    {
      "type": "bridge",
      "bridge": "podman-vlan",
      "isGateway": true,
      "ipMasq": true,
      "promiscMode": true,
      "ipam": {
        "type": "host-local",
        "routes": [{ "dst": "0.0.0.0/0" }],
        "ranges": [
          [
            {
              "subnet": "172.20.0.0/16",
              "gateway": "172.20.0.1"
            }
          ]
        ]
      }
    },
    {
      "type": "portmap",
      "capabilities": {
        "portMappings": true
      }
    },
    {
      "type": "firewall"
    },
    {
      "type": "tuning"
    }
  ]
}

Thanks in advance.

185
0 + 0
Score:0
cpl avatar Server
us flag
cpl

It was an issue with Docker existing on that VM and iptables.

Checking iptables -L on the VM

Chain FORWARD (policy DROP)
target     prot opt source               destination
DOCKER-USER  all  --  anywhere             anywhere
CNI-FORWARD  all  --  anywhere             anywhere

Docker took priority. So either configure the rule to forward the right interface to CNI/Podman/Docker, or whatever is needed.

In my case, removing Docker was an option and that fixed everything.

0 + 0
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.