My hardware is a brand new Asus RS72A-E12-RS24U server with two AMD EPYC 9654 CPUs. I'm having a problem with the wired network adapter. The NIC is a Broadcom N210TP. It's a low-profile OCP 3.0 10g card with two RJ-45 ports. I have a completely fresh installation of Ubuntu 22.04.2 LTS, straight from a USB stick with software updates and third party drivers accepted.
I have the default network configuration, which is automatically managed by NetworkManager. After installing all the software updates and booting, the system recognizes the 10G interface, successfully connecting at 10,000 Mb/s. But within two minutes of booting up, the system, the link suddenly disconnects. I can still access the system using a 1G network connection built onto the motherboard. (I use DHCP to assign different IP addresses to the two interfaces based on their MAC IDs, so I'm not causing the problem by having two devices with the same IP address on the network). Based on this fact, I think NetworkManager is still working correctly.
I've read a number of other articles on askubuntu.com and elsewhere online with similar problems, especially for Broadcom NICs. I've tried the most commonly recommended solutions, in particular installing linux-modules-extra. I'm running kernel version 5.19.0-43-generic and the output of
sudo dpkg -s linux-modules-extra-$(uname -r) is Package: linux-modules-extra-5.19.0-43-generic Status: install ok installed
The best evidence of the problem I have comes from running the command $ sudo lshw -c network which produces the output
*-network:0 UNCLAIMED
description: Ethernet controller
product: BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller
vendor: Broadcom Inc. and subsidiaries
physical id: 0
bus info: pci@0000:41:00.0
One other clue I got came from a session at the physical terminal in tty mode. I had a blinking cursor at the command prompt, and suddenly a text message appeared. I took a photo of it with my phone and the text read:
bxnt_en 0000:41:00.0 (unnamed net_device) (uninitialized) Error (timeout: 500015)
This was in another session where the 10G adapter had first been recognized for a few minutes. I believe other work on this problem that bnxt_en is the kernel driver for a number of Broadcom wired network adapters.
I'm hoping the askubuntu community might have some ideas about what's going wrong here and how to fix it. My first guess is that the driver is OK, because otherwise how could the NIC be recognized and successfully connect for a few minutes before it disconnects? My hypothesis is that it's a bad piece of hardware that needs to be replaced. Are there other software configuration or driver related troubleshooting steps I've missed? Do I have enough evidence to send it back to Broadcom and request an RMA?
Thanks,
-Michael
p.s. I've been an Ubuntu user since 2018 and the community has been a huge resource in learning and solving problems, especially posts on this site.
