Join Log in
Score:0
avatar Ubuntu

Ubuntu 20.04 Mellanox MT27800 40Gbit Capacity

cn flag
Sean Ferguson

Server Specifications

  • Platform: HPE ProLiant DL380 Gen9 BIOS v2.90
  • Processors: Dual Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz
  • Network Interface: HPE InfiniBand EDR/Ethernet 100Gb 2-Port 841QSFP28 Mellanox MT27800 Firmware 16.31.1200

When I run a lshw -class network I get the output below:

*-network:1
       description: Ethernet interface
       product: MT27800 Family [ConnectX-5]
       vendor: Mellanox Technologies
       physical id: 0.1
       bus info: pci@0000:84:00.1
       logical name: ens5f0
       version: 00
       serial: 94:40:XX:XX:XX:XX
       capacity: 40Gbit/s
       width: 64 bits
       clock: 33MHz
       capabilities: pciexpress vpd msix pm bus_master cap_list ethernet physical 1000bt-fd 10000bt-fd 25000bt-fd 40000bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=mlx5_core driverversion=5.4-1.0.3 duplex=full firmware=16.31.1200 (HPE0000000009) ip=10.X.X.X latency=0 link=yes multicast=yes
       resources: irq:17 memory:96000000-97ffffff memory:39ffae00000-39ffcdfffff

You can see that the Capacity is 40Gbit/s, however when I run ethtool I do see the Speed as 100000Mb/s. The Cisco Nexus Switch also shows the link speed at 100Gbps.

The HPE iLO Device inventory shows that the 100Gbps card is running at x16 can anyone help me figure out why I am still be bottle necked at 40Gbit/s?

Is there an issue with the PCI configuration or the auto negotiation?

141
0 + 0
hp
jtessier72 avatar
cn flag
jtessier72
You might check what ethtool is reporting. sudo ethtool ens5f0 and also sudo ethtool -m ens5f0
0
Reply
jtessier72 avatar
cn flag
jtessier72
notice that the capabilities line in lshw lists only up to 40000bt-fd. I note that you mentioned ethtool says 100000, it would be interesting to see what link modes ethtool says and also -m shows the optic info. I have a similar card, but driver & firmware are older and lshw doesn't list capacity at the moment.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.