Score:2

Route traffic through physical interface

cn flag

I have two ethernet interface on a linux machine. The both have the following IP addresses: Interface A: 10.0.1.5/24 Interface B: 10.0.2.5/24

Both interfaces are connected to a router that routes between the two networks.

Now I want to start an iperf3 measurment where I bind the server to Interface A and the client to Interface B.

The problem is that the traffic does not leave the physical interface but instead only flows through the kernel resulting in a speed that's not even possible on the link.

What can I do to force the traffic through the physical interface? ip_forward is disabled.

Thanks.

Score:1
cl flag
A.B

It would be very difficult (but not impossible) to have a Linux system, using a single routing stack, perform multi-homed routing of packets from itself to itself without using the lo interface.

But it's very easy on Linux to create additional network stacks to simulate multiple systems within one system: by using network namespaces.

Here one can relinquish one of the two NICs to a new network namespace which will be a peer to the initial (host) network namespace. They won't communicate directly, but only through the external router. Let's assume interfaces are really named A and B and that the router uses addresses 10.0.1.1/24 + 10.0.2.1/24.

  • create a new network namespace with additional facilitated management when using iproute2 tools (under the hood, namespace pseudo-files are mounted to keep the resource around without a process, etc.).

    ip netns add sideB
    
  • move interface B to the new network namespace:

    ip link set dev B netns sideB
    

    note: wireless interfaces require using the iw command instead.

  • configure the new network namespace:

    All of its network settings are lost when an interface changes namespace (both on the host where the interface disappears triggering addresses disappearing triggering routes disappearing, and on the new network namespace):

    ip -n sideB link set dev B up
    ip -n sideB address add 10.0.2.5/24 dev B
    

    Not needed but in case iperf3 gets confused, have a functional loopback interface:

    ip -n sideB link set lo up
    
  • configure routes between the two sides (initial/host namespace might have it already through the default route, but let's be explicit)

    ip route add 10.0.2.0/24 via 10.0.1.1 dev A
    
    ip -n sideB route add 10.0.1.0/24 via 10.0.2.1 dev B
    

Now one can run (in two terminals):

iperf3 -s

ip netns exec sideB iperf3 -c 10.0.1.5

with the client running in the new network namespace.

Once measuring is over, deleting the network namespace will return the NIC to the host namespace. But if any process is left by mistake using this network namespace (something like ip netns exec side B setsid sleep 9999) the ip netns delete sideB below will only remove the namespace from the view of the iproute2 tools, but not actually remove the namespace until the process ends, leaving the NIC difficult to recover back. So better move it back first:

ip -n sideB link set dev B netns 1
ip netns delete sideB

where 1 means PID 1's network namespace: the initial/host namespace. Additionally this allows to perform the experiment inside a container having been given physical NICs (because there 1's network namespace is the container's namespace, instead of the actual host's namespace) or the physical NIC would reappear on the host and be lost forever to the container if there's no access to the host.

If there's no tool (like udev + ifupdown's allow-hotplug or NetworkManager) detecting the apparition of the NIC, it will again have to be configured again on its return.

cn flag
Thanks! Unfortunately I cannot upvote yet.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.