Join Log in
Score:-1
Henry avatar Server

Onboard ethernet versus Intel NIC

cn flag
Henry

I would need some recommendation about the benefits of using a dedicated Intel Ethernet card versus simply using the integrated Ethernet of the motherboard. The typical use case would be the following:

  1. Payload of around 100 KB per message,
  2. Frequency of around 1000 messages per second maximum,
  3. Ideally, the handling of these messages should consume CPU time as minimum as possible (the CPU will be used for real time computation in a parallel process),
  4. Linux based server.

For the integrated Ethernet solution, I could have access to a mainstream motherboard (not server grade) equipped with an (integrated) Intel I225V or a Realtek RTL8125BG.

For the solution with an external NIC, I was thinking about the Intel X550-T1.

So, which solution would be better ?

In the case where a dedicated card is better for my use case, what PCI-express port should be favored ? Should the port used be as close as possible to the CPU ?

199
0 + 0
nic
Henry avatar
cn flag
Henry
To the person who downgraded the question, could you please explain why ?
0
Reply
us flag
Tero Kilkanen
In the past I have experienced reliability issues with onboard NICs, while an add-on Intel PCI card has worked perfectly.
0
Reply
Henry avatar
cn flag
Henry
@TeroKilkanen Thanks for the information. Could you please detail in an answer what kind of issues you experienced ?
0
Reply
us flag
Tero Kilkanen
Performance issues due to driver quality and crashes.
0
Reply
Score:1
Vipin Varghese avatar Server
cn flag
Vipin Varghese

In order to compare onboard vs PCIe NIC, one has to use the same ASIC model with the same firmware to eliminate external factors. With this assumption let us take into account for X550-T1 ASIC. Some of the factors which affects onboard vs PCIe card are

  1. Number of dedicated PCIe lanes.
  2. NUMA CPU threads accessing the NIC.
  3. Firmware available for onboard NIC (certain motherboard vendors will ship custom firmware only).
  4. read and write buffer size from the PCI enumeration.
  5. low latency mode (example -a 0000:88:00.0,rx_low_latency=1).

Note: all other factors like memory, CPU, disk, I/O, UNCORE are all system or CPU related and does not affect onboard vs external PCIe NIC.

0 + 0
Henry avatar
cn flag
Henry
Thanks for the insight. From what you said, considering there are many unknowns about the implementation of the onboard network chip, both software and hardware, would it be reasonable to consider that, on average, it is better to use an external card, like the Intel one cited above, where the driver and firmware are known, and where the connection can be chosen (in the sense a fast PCIe port) ?
0
Reply
Vipin Varghese avatar
cn flag
Vipin Varghese
@Henry, there is the motherboard that supports X710 and X550 as onboard NIC ASIC also. Hence comparison is to be done against similar family external NIC, since all other factors remain the same. A good example is Intel Xeon-D where integrated 10Gbps and 25Gbps are present. Based on the use case it always better to explore as price and PCIe slots can be saved with integrated NIC with SoC.
0
Reply
Vipin Varghese avatar
cn flag
Vipin Varghese
@Henry if you find the answer useful please accept and upvote.
0
Reply
Henry avatar
cn flag
Henry
Sorry, I had already accepted the first answer and unfortunately I do not have enough reputation to upvote your answer.
0
Reply
Vipin Varghese avatar
cn flag
Vipin Varghese
@Henry thanks for the update. I hope you are clear in the differences between onboard vs external for the same ASIC query?
0
Reply
Score:1
Anton Danilov avatar Server
cn flag
Anton Danilov

There isn't a difference in latency between onboard and dedicated NIC. Both of these use the PCI-express. Likely the dedicated NIC will be connected to internal PCI-e controller in CPU. Onboard integrated NIC can be connected to the chipset integrated conntroller, not directly to the CPU. But the difference in latencies of transmit data from NIC to memory is almost absent.

If you really care about performance and latency of processing, check next points:

  • NICs uses the DMA to transmit data to the main memory, so use the memory with highest frequency
  • Newer version of PCI-e is better (more speed - less time for transmit data)
  • CPU freq is more important than number of cores.
  • In NUMA systems the data should be processed at a local node.
  • Don't forget disable the power-saving features
  • Use the software with the kernel-bypass like Intel DPDK
  • Read about Smart-NIC or FPGA-based DPU to offload the processing
0 + 0
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.