Join Log in
Score:2
Hubert Oliver avatar Server

My network suddenly completely stopped working on my Debian 11 server

sk flag
Hubert Oliver

I just recently set up a Debian 11 server as a host with several KVM guests. I have two physical interfaces on different LANs and set up two different bridges to connect the guests to the LANs that they need, with their own individual IPs. This was working perfectly when I shut the server down so that I could add drives to it. While the server was down (to add drives to it), I changed the IPs that were assigned by DHCP to the bridge interfaces. Once I powered it back on (three days later), the network would not work at all. I could not ping the gateway, I could not do DNS lookups, I could not SSH into the server, and I could not even ping the guests from inside the server itself. I have tried everything I can think of, restarted the networking service a dozen times, restarted the server another half dozen times, and nothing works. The interfaces are all getting their correct IPs via DHCP, but I cannot get the network to work at all. I don't know what happened. Changing the IPs for the bridge interfaces via DHCP should not have had any effect. I have also updated and restarted the firewall during my troubleshooting. The rest of my network is fine. The only way I am able to get any network at all is to blow away all of the interfaces but one in /etc/network/interfaces and restart the server. Obviously, this is not going to work in production. Does anyone have any ideas? I am at my wits end. I will include all the information I can think of:

/etc/network/interfaces

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# Onboard Ethernet 1 (top)
allow-hotplug eno1
auto eno1
iface eno1 inet dhcp
# dhcp

# Onboard Ehternet 2 (bottom)
auto eno2
allow-hotplug eno2
iface eno2 inet dhcp


#Bridge setup
auto br0
iface br0 inet dhcp
    bridge_ports eno1
    bridge_stp off
    bridge_maxwait 0
    bridge_fd 0

auto br1
iface br1 inet dhcp
    bridge_ports eno2
    bridge_stp off
    bridge_maxwait 0
    bridge_fd 0

brctl_show

bridge name bridge id       STP enabled interfaces
br0     8000.0258ab8fb9c5   no      eno1
                            vnet0
                            vnet3
                            vnet5
br1     8000.62ba805c2dfd   no      eno2
                            vnet1
                            vnet2
                            vnet4
int0        8000.525400acd0ee   yes

ip address

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:0a:cd:23:18:a4 brd ff:ff:ff:ff:ff:ff
    altname enp68s0
3: enp69s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:0a:cd:23:18:a5 brd ff:ff:ff:ff:ff:ff
4: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0 state UP group default qlen 1000
    link/ether 90:b1:1c:53:48:8d brd ff:ff:ff:ff:ff:ff
    altname enp6s0f0
    inet 192.168.0.130/24 brd 192.168.0.255 scope global dynamic eno1
       valid_lft 2350sec preferred_lft 2350sec
5: eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br1 state UP group default qlen 1000
    link/ether 90:b1:1c:53:48:8e brd ff:ff:ff:ff:ff:ff
    altname enp6s0f1
    inet 192.168.1.130/24 brd 192.168.1.255 scope global dynamic eno2
       valid_lft 2349sec preferred_lft 2349sec
6: idrac: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 90:b1:1c:53:48:90 brd ff:ff:ff:ff:ff:ff
7: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 02:58:ab:8f:b9:c5 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.118/24 brd 192.168.0.255 scope global dynamic br0
       valid_lft 2350sec preferred_lft 2350sec
    inet6 fe80::58:abff:fe8f:b9c5/64 scope link 
       valid_lft forever preferred_lft forever
8: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 62:ba:80:5c:2d:fd brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.2/24 brd 192.168.1.255 scope global dynamic br1
       valid_lft 2351sec preferred_lft 2351sec
    inet6 fe80::60ba:80ff:fe5c:2dfd/64 scope link 
       valid_lft forever preferred_lft forever
9: int0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:ac:d0:ee brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global int0
       valid_lft forever preferred_lft forever
10: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UNKNOWN group default qlen 1000
    link/ether fe:54:00:30:72:02 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc54:ff:fe30:7202/64 scope link 
       valid_lft forever preferred_lft forever
11: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br1 state UNKNOWN group default qlen 1000
    link/ether fe:54:00:30:72:12 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc54:ff:fe30:7212/64 scope link 
       valid_lft forever preferred_lft forever
12: vnet2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br1 state UNKNOWN group default qlen 1000
    link/ether fe:54:00:30:72:13 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc54:ff:fe30:7213/64 scope link 
       valid_lft forever preferred_lft forever
13: vnet3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UNKNOWN group default qlen 1000
    link/ether fe:54:00:30:72:05 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc54:ff:fe30:7205/64 scope link 
       valid_lft forever preferred_lft forever
14: vnet4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br1 state UNKNOWN group default qlen 1000
    link/ether fe:54:00:30:72:15 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc54:ff:fe30:7215/64 scope link 
       valid_lft forever preferred_lft forever
15: vnet5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UNKNOWN group default qlen 1000
    link/ether fe:54:00:30:72:04 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc54:ff:fe30:7204/64 scope link 
       valid_lft forever preferred_lft forever

ip route

default via 192.168.1.1 dev eno2 
192.168.0.0/24 dev eno1 proto kernel scope link src 192.168.0.130 
192.168.0.0/24 dev br0 proto kernel scope link src 192.168.0.118 
192.168.1.0/24 dev eno2 proto kernel scope link src 192.168.1.130 
192.168.1.0/24 dev br1 proto kernel scope link src 192.168.1.2 
192.168.122.0/24 dev int0 proto kernel scope link src 192.168.122.1 linkdown

I've also had the default route come up as 192.168.0.1 dev eno1, but it still doesn't work. I'd prefer that the default be 192.168.0.1, but right now, I just need it to work. The firewall (DNS and DHCP server) is at 192.168.0.1 and 192.168.1.1.

Does anyone have any ideas that could help me troubleshoot this?

1746
1 + 0
Score:3
Nikita Kipriyanov avatar Server
za flag
Nikita Kipriyanov

It is wrong to assign an IP address to a bridge member, by any method, be it DHCP or static address or anything else. If it has been working before that was just by accident, probably the NIC initialization sequence happened to be too fortunate and "connected"-type routes were set right so it worked.

Set eno1 and eno2 configuration method to manual:

iface eno1 inet manual

iface eno2 inet manual

You don't need anything else for these interfaces (nor auto or allow-hotplug); bridges owning them will actually pull them up during initialization.

Assign IPs to bridges only, probably static configuration is suited better for the server than DHCP assigned, but that's entirely your decision.

Also, notice that brctl (and the whole bridge-utils package) is obsolete and the information it shows is redundant. We can see what is member of what by looking at the master property of other interfaces in the ip addr (or ip link) output. For more detailed information and manipulations with bridges use the bridge utility bundled with the iproute2 (which you already have, ip belongs there too).

+ 1
Hubert Oliver avatar
sk flag
Hubert Oliver
Thanks, that helped a lot! I can get to and from the host now. It's strange, the IPs are no longer listed on the virtual interfaces on the host, but I can ping and SSH to them just fine. Not what I expected, but it is working so far. Thanks again!
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.