Score:2

Server

load balancing over single NIC to overcome per-connection limit

mohotmoz

2/23/23, 12:06 PM

I have a strange setup where the ISP provides a very fast connection (10 Gbps) but limits each connection to 50 Mbps. This is fine for multi-threaded applications where I can just boost the number of threads. But I would like to solve this problem for single-threaded applications as well. I am running Linux and this is all TCP traffic (has to be) - I feel like there is a good way to do this using iptables but this is a little out of my depth. I am limited to one NIC card on the device. Is there a way to make a load balancer that just create multiple connections over the same NIC and then round robin the packets? (Basically I am already doing this in software for multithreaded apps.. but want to do it at the OS level).

problem solved what happened was.. not limited by the ISP at all. I was trying to connect to devices that have a fixed TCP window size. changing those devices to dynamic TCP window size increased the per-connection throughput to the max. more here:

https://en.wikipedia.org/wiki/Bandwidth-delay_product

190

0 + 0

nat

iptables

tcp

Nonny Moose

2/23/23, 10:11 PM

Did they say why they did that?

mohotmoz

2/24/23, 11:49 AM

This is a hosted environment - you can access your servers over the public internet for relatively cheap. But our org has opted for dark fiber. We actually pay for 10 Gbps. So I can get 10 Gbps... with a zillion threads. I think the ISP didn't realize how many people would opt for dark fiber so they underprovisioned it big time.. and now they have to speed limit people on a per-connection basis (without violating their total speed limit that we paid for.. thats in the contract). This whole thing is really expensive.. trying to get our moneys worth : )

Score:2

Server

Zac67

2/23/23, 12:28 PM

I don't think there's a point.

Of course, you can use multiple source IPs (whether from multiple hosts, multiple NICs or even a single NIC) but then again, you'd likely NAT all of those to the public IPv4 address you've got - so there's no difference from the outside. You can just as well just use multiple threads.

But obviously, you can just try binding multiple source IP addresses to a single NIC, not much to it.

0 + 0

mohotmoz

2/24/23, 11:45 AM

Thanks for the quick reply - I am going to try to create a bunch of IPs on my host (same NIC), then use iptables to round robin to them.. and then NAT out the other end (all on the same host). Maybe this will somehow trick the system into thinking they are separate connections. I am not sure HOW they are tracking.. but it can be the exact same source and exact same destination.. as long as it is in a different thread, it gets its own bandwidth!

Zac67

2/24/23, 2:00 PM

Likely, multiple threads / socket connections are all that you require. Throwing in different IP addresses *before* NAT can't really make a difference (unless they're required to make the application use ECMP).

Score:1

Server

Chopper3

2/23/23, 12:51 PM

There's two ways this has been limited - on a per-VM basis or on a per-vNIC-basis. If it's on a per-VM basis there's nothing you can do other than threaten to move to a different provider or actually move providers, obviously the former might not work. If it's on a per-vNIC basis then you could either add more yourself if that's possible or again ask the provider to add them, and again if they refuse threaten to leave.

Ultimately you have a business need that it sounds like isn't being met by your provider, so be prepared to pay a little more to get what you need.

0 + 0

mohotmoz

2/24/23, 11:43 AM

Thanks so much for the quick reply - it is really weird for sure. They are somehow tracking the each connection and then limiting it - but they don't care if the source and destination are completely identical. My multi-threaded code is literally doing the exact same thing in parallel (same source, same destination - in this case over HTTPS but have tried others and it doesn't matter). I guess I'm wondering if there is a way to abstract this out to the OS level so every TCP connection turns into.. say.. 50 of them.. as opposed to having to re-tool each piece of software to do this internally.

mohotmoz

2/24/23, 11:54 AM

Said differently.. it is not on a per-VM or a per-vNIC basis... it is definitely on a per-connection basis... I am just not sure HOW this is tracked and if there is an elegant way to get around it : ) We are guaranteed a certain speed (it is dark fiber).. see my comment to the other poster above.. but they are limiting our per-connection speed I think to avoid congestion. I have tested the exact same setup with a different ISP that doesn't thread limit (we have redundant ISPs.. redundant data centers..etc) and it is like 50x faster per thread.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: load balancing over single NIC to overcome per-connection limit

TH: โหลดบาลานซ์ผ่าน NIC เดียวเพื่อเอาชนะขีดจำกัดต่อการเชื่อมต่อ

RO: echilibrare a sarcinii pe un singur NIC pentru a depăși limita pe conexiune

RU: балансировка нагрузки по одному сетевому адаптеру для преодоления ограничения на количество подключений

VI: cân bằng tải trên một NIC để vượt qua giới hạn trên mỗi kết nối

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.