Join Log in
Score:1
avatar Server

Load Balancing HTTP/2 GRPC persistent connections

in flag
Sam

Let's take a look at this scenario.

In a traditional LB model, we would have the LB (wether a reverse proxy or not) farm out req/rep to application level servers. This model is fine for the generic req/rep style of communication, but what happens when we have persistent connections.

Let's look at a HTTP/2 example.

Here we don't have a simple Req/Rep paradigm where i can simply send the request and load balance that request and get a response. That entire architecture is stateless (forget sticky sessions as that has implications not relevant in this example). What happens we have a persistent connection such as with HTTP/2.

I can't connect the HTTP/2 connection to the LB or else that LB will saturate extremely quickly and all client requests that don't already have a socket connected with that LB will be out of luck. If they are relying on DNS to point to that LB, lots of problems are afoot when we run out of socket space.

We need something to almost redirect the initial SYN request to another server that could be another LB or the application layer itself (forget about security/outside attacks).

Client sends request s1.com --> (hits DNS and returns ip .123)

Client then sends HTTP/2 connection request to .123 (this is an LB) but instead of connecting and passing onto the application layer, it sends back a response to "redirect .. similar to a 302 in HTTP" to ip .124 . IP .124 could be a different kind of LB or could be the application layer itself, where the actual socket is bound to for HTTP/2 communication.

The management of .123 and which nodes it points to could be managed with a variety of tools that currently exist, but in terms of this "HTTP/2 redirect to balance the socket load problem" I haven't been able to find a solution.

Is there a better way to handle this (once again not for internal services but for external facing requests) or is there a proxy/service/system that exists to solve this problem?

DSR is slightly different in that it solves the problem at a different layer and is really relevant when you have a lot of return traffic but doesn't solve the problem if traffic is bi-directional (also does not work on a lot of cloud providers). DSR makes a lot of sense for media streaming and heavy server response traffic.

Thank you for any insights or help you can provide on this topic!

How

126
0 + 0
in flag
Sam
i almost thought about making something in rust or erlang that basically accepts an HTTP1.x request(this request is coming from a browser) that accepts the socket and return a dnsname+token. And from that response setting up the HTTP/2 connection with that dns name and token info (for auth on the other server).
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.