Questions on 40GBe QSFP+ to 4X 10GBe SFP+ breakout cables

Kyungrae Kang

4/18/24, 6:34 PM

I run a small(~400cores) Linux cluster. I am using aggregated 10GBe ethernet ports(with Netgear XS728T), but now considering an upgrade to 100GB ethernet with QSFP28 ports(Mellanox SN2100).

As I don't want too much work on the upgrade, I want to use a QSFP+ to sfp+ breakout cable but I wish someone who knows better could check if the new layout would work.

  (WAN)
    |
    |      <- RJ45 cable
    |
<10GBe switch, RJ45/SFP+>
    |
    |      <- 40GBe QSFP+ to 10GBe SFP+ breakout cable
    ||
<100GBe switch, QSFP28>
    ||
    ||      <- QSFP28 cable
    ||
(compute nodes)

The question is,

Would the internet directly established on the compute nodes with the above setup?

(so that I can reach the repos, webpages and stuff)
Is any other specific setting required for either/both of the switches?

Thanks in advance

1 + 0

networking

ethernet

switch

sfp

Score:1

Server

Mintra

4/19/24, 3:47 PM

Your diagram doesn't show a router, only switches, so in answer to your question 1 I am not sure how you currently get internet access? If we assume that what you have labelled as (WAN) is actually the inside interface of a router then yes you should continue to be able to get internet access on the nodes as they will continue to have L2 connectivity to the router.

As for question 2, you will have to configure the Mellanox switch to be able to use a breakout cable. Refer to the switch documentation but I believe the command to run from within the QSFP28 interface context is module-type qsfp-split-4 - then you will get 4 sub-interfaces created which you need to set to 10G with e.g. interface eth 1/1/1 speed 10G no-autoneg force

+ 1

Kyungrae Kang

4/19/24, 5:38 PM

Thanks! Your assumption is right. I omitted the details inside but that WAN is the network port from a router. As for the command, I will look into it. Thanks again for your information

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Questions on 40GBe QSFP+ to 4X 10GBe SFP+ breakout cables

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.