Score:0

Why does my network stop working when adding a second IP to a second NIC in netplan and starting a docker container exposed on that nic's IP?

in flag

"Stop working" might be an overstatement, but parts of it do, and I don't know enough about what's happening to phrase it shorter for a title.

Basically, I have

  • A machine with 2 nic srunning one VM and a bunch of Docker containers. NIC#1 has IP 10.0.30.21 and NIC#2 was recenlty set up and assigned 10.0.30.22.
    • One VM running Home Assistant OS (IP: 10.0.30.61:8123, and served by NGINX at hass.my_domain.tld)
    • A bunch of Docker containers, incl. of (potential) interest:
      • NGINX (linuxserver/swag) (using ports 80 and 443)
      • deCONZ (using ports 8080 and 8443)
      • Z-WaveJS2MQTT Server (using port 8091 and 3000)

All docker containers of interest expose ports on 10.0.30.21.

Until now, I have only used the one nic, but for unrelated reasons, I want to move my unifi controller to a docker container. Since it uses some ports already in use on the 10.0.30.21 IP (e.g. the ones used by deCONZ), I wanted to set up the second NIC and use a different IP.

So, I added the second interface to my /etc/netplan/00-installer-config.yaml:

network:
  ethernets:
    eno1:
      dhcp4: true
    eno2:            # <-- These two lines are what I added
      dhcp4: true    # <-- These two lines are what I added
  version: 2
  bridges:
    br0:
      dhcp4: yes
      interfaces:
             - eno1
      parameters:
        stp: true

It seems at first to work fine, the new mac shows up and I can assign IP and all that in the dhcp server.

For the unifi controller, I used the port bindings that should tie everything to the "new" 10.0.30.22 IP (from the unifi-controller section in my docker-compose):

    ports:
      - 10.0.30.22:8443:8443
      - 10.0.30.22:3478:3478/udp
      - 10.0.30.22:10001:10001/udp
      - 10.0.30.22:8080:8080
      - 10.0.30.22:1900:1900/udp #optional
      - 10.0.30.22:8843:8843 #optional
      - 10.0.30.22:8880:8880 #optional
      - 10.0.30.22:6789:6789 #optional
      - 10.0.30.22:5514:5514/udp #optional

But when I then start the unifi-controller docker container (linuxserver/unifi-controller), a few things happen:

  1. The Home Assistant VM can no longer access deCONZ or Z-waveJS2MQTT services
    • The error logs just say that they are not available at their IP (10.0.30.21)
    • I can still access both of their web UIs at local IP:port (10.0.30.21:8080 and 10.0.30.21:8091), and I can control devices through this web UI (so it seems fully functional)
  2. The Home Assistant VM web UI is no longer accessible at hass.my_domain.tld despite:
    • The web UI being reachable and functional at the local IP (10.0.30.61:8123)
    • No errors or warnings in NGINX logs
    • No errors or warnings in Home Assistant logs
    • Maybe errors in docker logs deconz, I'm not sure if related.
      • deCONS: Unbind response failed with status 0x88 for 0x84fd27fffe70d3ed ep: 0x01 cluster: 0x2000
  3. I did not loose FQDN access to any other services served by NGINX, which include both services on that machine, and on other machines, e.g.:
    • Nextcloud (local IP 10.0.30.67)
    • DokuWiki (local IP 10.0.30.21)
    • TrueNAS (local IP 10.0.30.27)
    • PiHole (local IP 10.0.30.43)
    • etc.

When I then kill the unifi-controller, noting changes, there's still no access. When I remove the unifi-controller section from my docker-compose, and completely reboot the system, nothing change. Still no access.

The only way I could restore was the remove the eno2 lines from the netplan file, then reboot, and everything came back as before.

What on earth am I doing wrong here?

(Ubuntu Server LTS 22.04)

David avatar
cn flag
What version of Ubuntu are you using. Please add it to the question not in a comment.
Aephir avatar
in flag
Sure, added to the above.
ru flag
Two DHCP entries will result in changing network gateways which breaks your internet. Whichever DHCP entry is *second* will result in that being your default Internet route and break your network access to the Internet, etc. because of dual DHCP. At least *one* of those interfaces needs to be static and not set as the default route.
Aephir avatar
in flag
So it would be enough simply to change the `eno2` lines above to `dhcp: false` and add and `addresses: ` key with e.g. `- 10.0.30.22/24` (he asked, afraid to try due to the fury of family with network trouble)?
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.