Nginx can't accept TLS handshake when client is not sending TLS extensions

Z Wang

5/11/23, 4:10 PM

Nginx is configured as load balancer and has a server certificate installed to terminater TLS handshake for MQTT connections. It works when TLS extensions are included in clientHello during the handshake but there are other clients running old code who don't send any TLS extensions in clientHello. In that case I found Nginx will throw errors as in

SSL_do_handshake() failed (SSL: error:14201044:SSL routines:tls_choose_sigalg:internal error) while SSL handshaking, client: x.x.x.107, server: 0.0.0.0:8883

Here is Wireshark capturing

It looks to me Nginx is expecting at least the signature alorightms extension. My question is if possible Nginx can be configured to accept TLS handshake when client is not using extensions?

Nginx stream config is here (only show TLS related):

upstream brokers{
    server 127.0.0.1:18831;
    server 127.0.0.1:18832;
    server 127.0.0.1:18833;

    zone tcp_mem 64k;
    hash $mqtt_client_id consistent; 
}
server {
    listen 8883 ssl;  
    preread_buffer_size 1k;

    ssl_certificate /etc/nginx/certs/server.pem;
    ssl_certificate_key /etc/nginx/certs/server.key;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_session_cache shared:SSL:128m;  
    ssl_session_tickets on;
    ssl_session_timeout 8h;
    ssl_verify_client off;
    ssl_prefer_server_ciphers off;    
 
    proxy_pass  brokers;
    proxy_connect_timeout 5s;
    access_log /var/log/nginx/mqtt_access.log mqtt;
    error_log /var/log/nginx/mqtt_error.log debug;  
}

Here is the wireshark captureing when clients send extensions in clientHello:

1432

0 + 0

ssl

nginx

extension

Z Wang

5/23/23, 5:45 AM

It turned out the openssl default security level prohibits RSA keys less 2048 bits in the ciphers. After adjusting the security level, the problem is resolved.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Nginx can't accept TLS handshake when client is not sending TLS extensions

TH: Nginx ไม่สามารถยอมรับ TLS handshake เมื่อไคลเอนต์ไม่ส่งส่วนขยาย TLS

RO: Nginx nu poate accepta acordul TLS atunci când clientul nu trimite extensii TLS

RU: Nginx не может принять рукопожатие TLS, когда клиент не отправляет расширения TLS

VI: Nginx không thể chấp nhận bắt tay TLS khi khách hàng không gửi tiện ích mở rộng TLS

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.