I've been trying to manually decrypt some TLS 1.3 traffic for educational reasons, and have stumbled across a roadblock. So far, I've been able to complete most of the key schedule, including deriving the correct handshake traffic secrets from the ECDHE keys. However, I haven't been able to find much information about how the ciphertext and authentication tag are formatted within Application Data messages.
Here is the full record.
1703030017bf687d10e2f209661418d92aaf3626dfe5670f3127d6ed
And here is the server handshake traffic secret:
475a0f3b5a86d4797e24545c2eb6388de9507e5e0bc367c74c3e78e0d8033f34b721a2e3a1d89ec5990287954402fc43
Wireshark tells me that decrypting this message yields the plaintext 08 00 00 02 00 00, corresponding to the Encrypted Extensions message.
From what I can tell, the "associated data" should be the bytes 17 03 03 00 17, and the rest of record is ciphertext + authentication tag. RFC 8446 says that the TLS_AES_256_GCM_SHA384 cipher suite is defined in RFC 5116, but RFC 5116 says:
This document does not specify any particular encoding for the AEAD
inputs and outputs, since the encoding does not affect the security
services provided by an AEAD algorithm.
Yet all my attempts to decrypt the data have failed to authenticate, which leads me to two possibilities:
- I am deriving the traffic keys wrong. I know that my HKDF-Expand-Label routine works since all other secrets are derived correctly and match the values logged by OpenSSL. However, admittedly I'm not 100% sure what the correct key and IV length are. Currently I am using a 32-byte key and 12-byte IV. I don't know if that's what TLS 1.3 uses.
- I am reading the authentication tag/ciphertext wrong. I haven't been able to find any references on what the proper format is, so all of my attempts in this area have just been guesses. In particular, I'm not sure how long the authentication tag is for this cipher suite, though googling suggests that it's 12 bytes.
What am I doing wrong?
