I have a WEB application deployed to Tomcat server. I connect to it with Chrome browser with HTTPS but I have decrypt error during TLS handshaking on the client side after ServerKeyExchange.
Certificates (3 levels):
Server certificate, signed by...
CA certificate, signed by...
Root certificate (self signed)
I validated certificates with openssl and they seem to be fine (chain.cer contains CA and root certificates):
$ openssl verify -verbose -CAfile chain.cer server.cer
server.cer: OK
If I test connection with OpenSSL I get error after client reads ServerKeyExchange:
openssl.exe s_client -CAfile chain.cer -showcerts -state -msg server.net:8443
output:
CONNECTED(00000004)
>>> ??? [length 0005]
16 03 01 01 4f
>>> TLS 1.3, Handshake [length 014f], ClientHello
01 00 01 4b 03 03 81 63 a4 15 45 bf 7f 9b 07 8f ...
<<< ??? [length 0005]
16 03 03 09 14
<<< TLS 1.3, Handshake [length 0055], ServerHello
02 00 00 51 03 03 60 ef d0 8b 1c d7 9a 78 2d d4 ...
<<< TLS 1.2, Handshake [length 07ee], Certificate
0b 00 07 ea 00 07 e7 00 07 e4 30 82 07 e0 30 82 ...
depth=2 O = Amadeus IT group SA, CN = amarootca2
verify return:1
depth=1 O = Amadeus IT group SA, CN = amacatech3
verify return:1
depth=0 C = FR, L = Nice, O = Amadeus Data Processing, OU = NIS, CN = nceiptapas04.nce.amadeus.net
verify return:1
<<< TLS 1.2, Handshake [length 00cd], ServerKeyExchange
0c 00 00 c9 03 00 17 41 04 82 07 58 e1 cd 42 40 ...
>>> ??? [length 0005]
15 03 03 00 02
>>> TLS 1.2, Alert [length 0002], fatal decrypt_error
02 33
34359738384:error:04091077:rsa routines:int_rsa_verify:wrong signature length:crypto/rsa/rsa_sign.c:132:
34359738384:error:1416D07B:SSL routines:tls_process_key_exchange:bad signature:ssl/statem/statem_clnt.c:2405:
---
Cannot client decrypt DH parameters sent by server? Why?
Here is Wireshark details from ServerKeyExchange:
Signature Algorithm: rsa_pss_rsae_sha256 (0x0804)
Signature Length: 128
I have another but properly working WEB application where I have the same Signature Algorithm but the Signature Length: 256. Or this length is irrelevant?