Getting CD on HAProxy TCP (Layer 4) connection for Windows RDS Gateway

I have a Windows RDS instance that users connect to over the internet, although we have placed an HAProxy using a TCP frontend/backend:

frontend front_https_rds
        bind XXX.XXX.XXX.XXX:443
        mode tcp
        timeout client 6h
        log global
        option tcplog
        option clitcpka
        default_backend back_https_rds

backend back_https_rds
        mode tcp
        timeout server 6h
        balance roundrobin
        log global
        option tcplog
        option tcp-check
        server srv-rdgateway pri.vat.e.IP:443 check

It's important to note, I am using the RD Gateway over HTTPS (RPC over HTTP), and not the standard RDP 3389.

The users are complaining about getting frequent disconnects. I am seeing CD (Client disconnects in the data phase) in the termination_state column. From the HAProxy logs:

Nov 23 18:14:24 srv-haproxy haproxy[215010]: XXX.XXX.XXX.XXX:55674 [23/Nov/2021:18:08:48.979] front_https_rds back_https_rds/srv-rdgateway 1/0/335916 425072 CD 20/20/19/19/0 0/0
Nov 23 18:14:24 srv-haproxy haproxy[215010]: XXX.XXX.XXX.XXX:55678 [23/Nov/2021:18:08:49.370] front_https_rds back_https_rds/srv-rdgateway 1/0/335526 1295 SD 19/19/18/18/0 0/0
Nov 23 18:14:35 srv-haproxy haproxy[215010]: XXX.XXX.XXX.XXX:56448 [23/Nov/2021:18:14:35.135] front_https_rds back_https_rds/srv-rdgateway 1/0/96 3978 SD 20/20/19/19/0 0/0
Nov 23 18:14:35 srv-haproxy haproxy[215010]: XXX.XXX.XXX.XXX:56451 [23/Nov/2021:18:14:35.461] front_https_rds back_https_rds/srv-rdgateway 1/0/91 3978 SD 21/21/20/20/0 0/0
Nov 23 18:14:36 srv-haproxy haproxy[215010]: XXX.XXX.XXX.XXX:56453 [23/Nov/2021:18:14:35.965] front_https_rds back_https_rds/srv-rdgateway 1/0/89 3978 SD 21/21/20/20/0 0/0
Nov 23 18:16:13 srv-haproxy haproxy[215010]: XXX.XXX.XXX.XXX:56447 [23/Nov/2021:18:14:34.997] front_https_rds back_https_rds/srv-rdgateway 1/0/98162 431661 CD 14/14/13/13/0 0/0
Nov 23 18:16:13 srv-haproxy haproxy[215010]: XXX.XXX.XXX.XXX:56450 [23/Nov/2021:18:14:35.385] front_https_rds back_https_rds/srv-rdgateway 1/0/97776 1295 SD 13/13/12/12/0 0/0
Nov 23 18:16:23 srv-haproxy haproxy[215010]: XXX.XXX.XXX.XXX:56746 [23/Nov/2021:18:16:23.423] front_https_rds back_https_rds/srv-rdgateway 1/0/86 3978 SD 14/14/13/13/0 0/0
Nov 23 18:16:23 srv-haproxy haproxy[215010]: XXX.XXX.XXX.XXX:56749 [23/Nov/2021:18:16:23.728] front_https_rds back_https_rds/srv-rdgateway 1/0/98 3978 SD 15/15/14/14/0 0/0
Nov 23 18:16:24 srv-haproxy haproxy[215010]: XXX.XXX.XXX.XXX:56751 [23/Nov/2021:18:16:24.232] front_https_rds back_https_rds/srv-rdgateway 1/0/92 3978 SD 15/15/14/14/0 0/0
Nov 23 18:16:57 srv-haproxy haproxy[215010]: XXX.XXX.XXX.XXX:56745 [23/Nov/2021:18:16:23.274] front_https_rds back_https_rds/srv-rdgateway 1/0/34020 230984 CD 14/14/13/13/0 0/0
Nov 23 18:16:57 srv-haproxy haproxy[215010]: XXX.XXX.XXX.XXX:56747 [23/Nov/2021:18:16:23.655] front_https_rds back_https_rds/srv-rdgateway 1/0/33641 1295 SD 13/13/12/12/0 0/0
Nov 23 18:17:07 srv-haproxy haproxy[215010]: XXX.XXX.XXX.XXX:56867 [23/Nov/2021:18:17:07.548] front_https_rds back_https_rds/srv-rdgateway 1/0/92 3978 SD 14/14/13/13/0 0/0
Nov 23 18:17:07 srv-haproxy haproxy[215010]: XXX.XXX.XXX.XXX:56870 [23/Nov/2021:18:17:07.872] front_https_rds back_https_rds/srv-rdgateway 1/0/92 3978 SD 15/15/14/14/0 0/0
Nov 23 18:17:08 srv-haproxy haproxy[215010]: XXX.XXX.XXX.XXX:56872 [23/Nov/2021:18:17:08.360] front_https_rds back_https_rds/srv-rdgateway 1/0/93 3978 SD 15/15/14/14/0 0/0

I assume something on the client is causing this, although I cant seem to identify anything. I added Keepalive in the frontend to the client with no change.

I am wondering if anyone has experienced this issue before, or anything I could do from HAProxy perspective to troubleshoot more?!

My Global config:

global
        log /dev/log    local0
        log /dev/log    local1 notice
        chroot /var/lib/haproxy
        stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
        stats timeout 30s
        user haproxy
        group haproxy
        daemon

        # Default SSL material locations
        ca-base /etc/ssl/certs
        crt-base /etc/ssl/private

        # Default ciphers to use on SSL-enabled listening sockets.
        # For more information, see ciphers(1SSL). This list is from:
        #  https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
        # An alternative list with additional directives can be obtained from
        #  https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy
        ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
        ssl-default-bind-options no-sslv3

defaults
        log     global
        mode    http
        option  httplog
        option  dontlognull
        timeout connect 5000
        timeout client  10800000
        timeout server  10800000
        errorfile 400 /etc/haproxy/errors/400.http
        errorfile 403 /etc/haproxy/errors/403.http
        errorfile 408 /etc/haproxy/errors/408.http
        errorfile 500 /etc/haproxy/errors/500.http
        errorfile 502 /etc/haproxy/errors/502.http
        errorfile 503 /etc/haproxy/errors/503.http
        errorfile 504 /etc/haproxy/errors/504.http