Score:0

Can't connect container with bridge to the internet using networkd

pm flag

i have a server running Ubuntu 20.04 LTS connected through one physical ethernet interface to the internet. My prodiver assigned me a static primary IP4 (i will use A.A.A.A here for this IP), so my systemd-networkd config file looked like this before (disabled netplan to work directly with systemd-networkd):

# /etc/systemd/network/20-enp7s0.network
[Match]
Name=enp7s0

[Network]
LinkLocalAddressing=ipv6
Address=A.A.A.A/32
Gateway=fe80::1
DNS=X.X.X.1
DNS=X.X.X.2

[Route]
Destination=0.0.0.0/0
Gateway=Y.Y.Y.Y
GatewayOnlink=true

My provider offers adding a additional IP address to my server, which is routed to the same interface as the primary IP. When adding this second IP to my interface i can ping it. Since i'm using systemd-nspawn containers i was thinking of using this additional IP to supply one of my container with an exclusive static IP4 (will use B.B.B.B here). This would be great to map DNS entries directly to a container on my server, while all other applications on the server still use the primary IP address.

So i started following the nice instructions from the Arch wiki on systemd-nspawn and systemd-networkd. I configured a bridge and moved all addressing from the physical interface to it:

/etc/systemd/network/br0.netdev

[NetDev]
Name=br0
Kind=bridge
MACAddress=xx:xx:xx:xx:xx:xx  # same as my phys. interface

/etc/systemd/network/20-br0.network

[Match]
Name=br0

[Network]
LinkLocalAddressing=ipv6
Address=A.A.A.A/32
Gateway=fe80::1
DNS=X.X.X.1
DNS=X.X.X.2

[Route]
Destination=0.0.0.0/0
Gateway=Y.Y.Y.Y
GatewayOnlink=true

/etc/systemd/network/20-enp7s0.network

[Match]
Name=enp7s0

[Network]
Bridge=br0

IP4-Forwarding is enabled:

$ sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1

I start my nspawn container with the following config:

/etc/systemd/nspawn/mycontainer.nspawn

[Network]
VirtualEthernet=yes
Bridge=br0

Inside the container (Debian 11 Bullseye) i enabled systemd-networkd and use the following config for networking:

# /etc/systemd/network/80-container-host0.network
[Match]
Name=host0

[Network]
Address=B.B.B.B/32
DNS=X.X.X.1
DNS=X.X.X.2

[Route]
Destination=0.0.0.0/0
Gateway=Y.Y.Y.Y
GatewayOnlink=true

This is the result of this configuration. On the host:

$ ip a

2: enp7s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0 state UP group default qlen 1000
    link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
3: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
    inet A.A.A.A/32 scope global br0
       valid_lft forever preferred_lft forever
6: vb-mycontainer@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UP group default qlen 1000
    link/ether yy:yy:yy:yy:yy:yy brd ff:ff:ff:ff:ff:ff link-netnsid 0

$ networkctl status -a
● 1: lo      [...]                                  
● 2: enp7s0                                                            
             Link File: /usr/lib/systemd/network/99-default.link       
          Network File: /etc/systemd/network/20-enp7s0.network         
                  Type: ether                                          
                 State: enslaved (configured)
                  Path: pci-0000:07:00.0                               
                Driver: igb                                            
                Vendor: Intel Corporation                              
                 Model: I210 Gigabit Network Connection                
            HW Address: xx:xx:xx:xx:xx:xx                              
                   MTU: 1500 (min: 68, max: 9216)                      
  Queue Length (Tx/Rx): 8/8                                            
      Auto negotiation: yes                                            
                 Speed: 1Gbps                                          
                Duplex: full                                           
                  Port: tp                                             
     Activation Policy: up                                             
   Required For Online: yes                                            
● 3: br0                                                                 
               Link File: /usr/lib/systemd/network/99-default.link       
            Network File: /etc/systemd/network/20-br0.network            
                    Type: bridge                                         
                   State: routable (configured)
                  Driver: bridge                                         
              HW Address: xx:xx:xx:xx:xx:xx                              
                     MTU: 1500 (min: 68, max: 65535)                     
           Forward Delay: 15s                                            
              Hello Time: 2s                                             
                 Max Age: 20s                                            
             Ageing Time: 5min                                           
                Priority: 32768                                          
                     STP: no                                             
  Multicast IGMP Version: 2                                              
    Queue Length (Tx/Rx): 1/1                                            
                 Address: A.A.A.A                                                  
                 Gateway: Y.Y.Y.Y (Juniper Networks)                  
                          fe80::1 (Juniper Networks)                     
                     DNS: X.X.X.1                                    
                          X.X.X.2                                                           
       Activation Policy: up                                             
     Required For Online: yes                                            
● 6: vb-mycontainer                                              
             Link File: /usr/lib/systemd/network/99-default.link
          Network File: n/a                                     
                  Type: ether                                   
                 State: degraded (unmanaged) 
                Driver: veth                                    
            HW Address: yy:yy:yy:yy:yy:yy                       
                   MTU: 1500 (min: 68, max: 65535)              
  Queue Length (Tx/Rx): 1/1                                     
      Auto negotiation: no                                      
                 Speed: 10Gbps                                  
                Duplex: full                                    
                  Port: tp                                      
               Address: fe80::xxxx:xxxx:xxxx:xxxx               
     Activation Policy: up                                      
   Required For Online: yes

$ ip route
default via Y.Y.Y.Y dev br0 proto static onlink

And inside my container:

# ip a
1: lo: [...]
2: host0@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether yy:yy:yy:yy:yy:yy brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet B.B.B.B/32 scope global host0
       valid_lft forever preferred_lft forever
    inet6 fe80::xxxx:xxxx:xxxx:xxxx/64 scope link 
       valid_lft forever preferred_lft forever

# networkctl status -a
● 1: lo           [...]
● 2: host0                                                                     
                     Link File: n/a
                  Network File: /etc/systemd/network/80-container-host0.network
                          Type: ether
                         State: routable (configured)
                    HW Address: zz:zz:zz:zz:zz:zz
                           MTU: 1500 (min: 68, max: 65535)
                         QDisc: noqueue
  IPv6 Address Generation Mode: eui64
          Queue Length (Tx/Rx): 1/1
              Auto negotiation: no
                         Speed: 10Gbps
                        Duplex: full
                          Port: tp
                       Address: B.B.B.B
                                fe80::xxxx:xxxx:xxxx:xxxx
                       Gateway: Y.Y.Y.Y
                           DNS: X.X.X.1
                                X.X.X.2
             DHCP6 Client DUID: DUID-EN/Vendor:0000ab117511f183668420370000

Feb 17 19:45:26 mycontainer systemd-networkd[25]: host0: Link UP
Feb 17 19:45:26 mycontainer systemd-networkd[25]: host0: Gained carrier
Feb 17 19:45:27 mycontainer systemd-networkd[25]: host0: Gained IPv6LL

# ip route
default via Y.Y.Y.Y dev host0 proto static onlink

Regarding all other settings i stick to the systems defaults. But its not working, i cant ping from the host to the guest, nor from the guest to the host, the internet or the gateway, just getting Destination Host Unreachable. So do i miss something here? I'm not really deep into networking and already spent a lot of time on this, but already apologize for some stupid mistakes i might made. Every clue is welcome. Thank you!

EDIT:

I had a look into the neighbors table:

Host:

$ ip neighbor
Y.Y.Y.Y dev br0 lladdr 84:c1:c1:76:ae:9b REACHABLE <- gateway
fe80::f80b:aff:fe80:d92 dev vb-mycontainer  FAILED
fe80::6c91:a7ff:fe1f:19a2 dev br0  FAILED
fe80::1 dev br0 lladdr 84:c1:c1:76:ae:9b router STALE
fe80::f80b:aff:fe80:d92 dev br0 lladdr fa:0b:0a:80:0d:92 STALE

Guest:

$ ip neighbor
fe80::7e10:c9ff:fe21:ed87 dev host0 lladdr 7c:10:c9:21:ed:87 router STALE
fe80::6c91:a7ff:fe1f:19a2 dev host0  FAILED
fe80::1 dev host0 lladdr 84:c1:c1:76:ae:9b router STALE

fe80::6c91:a7ff:fe1f:19a2 is the link-locale address of the virtual interface vb-mycontainer on the host. So there seems to be a connection problem between the guest and the host i assume?

Score:0
pm flag

Ok, i solved the problem on my own. I was missing to add a IP route in the bridge configuration on the host to my container:

# /etc/systemd/network/20-br0.network
[Match]
Name=br0

[Network]
LinkLocalAddressing=ipv6
Address=A.A.A.A/32
Gateway=fe80::1
DNS=X.X.X.1
DNS=X.X.X.2

[Route]
Destination=0.0.0.0/0
Gateway=Y.Y.Y.Y
GatewayOnlink=true

[Route]
Destination=B.B.B.B/32

And in the guest the gateway is the primary IPv4 address of the host (A.A.A.A/32):

# /etc/systemd/network/80-container-host0.network
[Match]
Name=host0

[Network]
Address=B.B.B.B/32
DNS=X.X.X.1
DNS=X.X.X.2

[Route]
Destination=0.0.0.0/0
Gateway=A.A.A.A
GatewayOnlink=true

Further more enabled systemd-resolved is necessary to get DNS resolution.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.