Why is "tcp-request content accept" frontend instruction is required for proper HAProxy SNI-based routing?

Pavel Klimov

11/25/23, 7:12 AM

I've recently tried to set up SNI-based routing on HAProxy for mongodb mongodb+srv protocol connection.

I made it working, but it wasn't until I put

tcp-request inspect-delay 5s
tcp-request content accept if { req_ssl_hello_type 1 }

in my frontend configuration that it started working properly

Without these (or with just one of these) I kept getting sporadic connection resets for around 70% of my requests. Note that sometime the connection did manage to get established.

According to http://cbonte.github.io/haproxy-dconv/2.0/configuration.html#4-tcp-request%20content

Content-based rules are evaluated in their exact declaration order. If no rule matches or if there is no rule, the default action is to accept the contents. There is no specific limit to the number of rules which may be inserted.

So it should have been working by default.

Either I'm missing some fundamental TCP concept or some specific Mongo connection detail. But i've noticed that almost everyone has the same rule defined when they go for SNI-based routing, so it must make sense for most folks.

Working HAProxy configuration:

    frontend fe_mongo-nonprod
      bind :27017
      tcp-request inspect-delay 5s
      tcp-request content accept if { req_ssl_hello_type 1 }

      acl mongoAtlas-01 req_ssl_sni -i host1.mongodb.net
      acl mongoAtlas-02 req_ssl_sni -i host2.mongodb.net
      acl mongoAtlas-03 req_ssl_sni -i host3.mongodb.net
      use_backend be_mongo-nonprod if mongoAtlas-01 || mongoAtlas-02 || mongoAtlas-03

    backend be_mongo-nonprod
      mode tcp

      acl mongoAtlas-01 req_ssl_sni -i host1.mongodb.net
      acl mongoAtlas-02 req_ssl_sni -i host2.mongodb.net
      acl mongoAtlas-03 req_ssl_sni -i host3.mongodb.net

      use-server server-01 if mongoAtlas-01
      use-server server-02 if mongoAtlas-02
      use-server server-03 if mongoAtlas-03

      server server-01 host1.mongodb.net:27017
      server server-02 host2.mongodb.net:27017
      server server-03 host3.mongodb.net:27017

It is also worth mentioning that HAProxy is deployed in Kubernetes (Google-managed cluster in GCP), has an istio-sidecar and is exposed via Internal LoadBalancer k8s service.

As stated the above config works perfectly well. What I'm interested in is why tcp-request content accept is absolutely required and connection is not [always] accepted by default here.

798

1 + 0

haproxy

sni

Score:3

Server

Pavel Klimov

11/28/23, 12:05 PM

I will try to answer my own question after giving it some thought.

So, HAProxy docs also have the following mention: http://cbonte.github.io/haproxy-dconv/2.0/configuration.html

If content switching is needed, it is recommended to first wait for a complete client hello (type 1), like in the example below.

Now, since the SNI data is also contained within the Client Hello section of the request ( https://www.rfc-editor.org/rfc/rfc6066#page-6 ) my assumption is the following:

In the case if Client Hello is spread across multiple TCP packets and no tcp-request content accept is configured, upon receiving the first packet HAProxy tries to route it and either of two things happen:

If default backend is configured the packet is forwarded to this backend.
If no default backend is configured (my case) the packet is dropped and connection is reset.

Also note that even in the first case described the server HAProxy points at in the backend section might require SNI just as well. So forwarding an incomplete request without the SNI indication will also result in the reset connection.

+ 2

Pavel Klimov

11/28/23, 12:06 PM

I will check this question periodically in case someone has a better explanation. If there's any - I will mark that as an accepted answer.

Mikko Rantalainen

1/27/24, 3:44 PM

The explanation in this answer seems plausible. It might be good idea to always configure default backend even if it's response is to simply log the connection and drop the connection. That way one would have at least better diagnostics for issues like this.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Why is "tcp-request content accept" frontend instruction is required for proper HAProxy SNI-based routing?

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.