Can a man in the middle tell what kind of encryption you're using from a key exchange?

gb flag

Encryption schemes typically are built on the idea that even if a man in the middle attacker knows exactly what kind of encryption you're using, they cannot decrypt your messages without the key.

There are a number of different mechanisms for two users to create or exchange keys without a man in the middle being able to deduce the keys. If a man in the middle attacker did not know in advance what kind of key exchange you were going to use, would that attacker be able to infer how you are exchanging keys?

Not that it would necessarily matter since that's the entire point of secure key exchanges. I'm mainly curious if, for instance, it would be possible to reverse engineer a proprietary key exchange protocol just from inspecting the packets that are being sent, even though you wouldn't be able to infer the keys themselves.

Steffen Ullrich avatar
kp flag
The title of the question ask about getting encryption from key exchange, the body instead asks about getting key exchange type from observing the key exchange - which are very different things. Apart from that even a more clear question is probably off-topic here and more on-topic on [].
fgrieu avatar
ng flag
In modern cryptography, the _"kind of encryption"_, and _"how you are exchanging keys"_, are assumed known to adversaries. Only keys are assumed unknown to adversaries (and further, in public key cryptography, public keys are assumed known to adversaries). That's [Kerckhoffs's (second) principle]( Please, [edit]( the question to make it in line with that (otherwise, the question might be closed as unclear). As an aside: a _private_ connection is not mandatory to resist MitM.
Marc Ilunga avatar
tr flag
Some authenticated key exchange protocols (like tls 1.3) encrypt part of the protocol for identity protection. One can extend that to also hide the ciphersuite. BUT, this is in no way a replacement for a carefully designed and secure protocol. As said in other comments, some good protocols are proven secure even when the attacker is given lots of power. Privacy of key exchange can be realized and analyzed as an additional goal.
se flag

There are subfields of cryptography research dedicated to make it hard to detect what ciphers and key exchange algorithms are being used. There are several rigorous approaches:

  1. Making the distribution of ciphertexts from two different symmetric ciphers indistinguishable from each other. This approach was introduced in Security of Symmetric Encryption against Mass Surveillance.
  2. Hide ciphertext A in a ciphertext B (from a different cipher) such that it is indistinguishable even given the keys and plaintext for B. Anamorphic Encryption: Private Communication against a Dictator
  3. Key Exchange that is indistinguishable from random + cryptographic stenography to hide key exchanges and ciphertexts in natural language distributions. Elligator: Elliptic-curve points indistinguishable from uniform random strings + Meteor: Cryptographically Secure Steganography for Realistic Distributions.
in flag

Generally it is possible to reverse-engineer a protocol by looking at messages. Commonly, protocols are build upon existing technology, such as X.509 certificates, ASN.1 BER/DER, binary encodings such as CBOR, textual encodings such as XML or JSON etc. These kind of of encoding structures contain a lot of information about what kind of information is being transmitted.

In most cases no effort is taken place to hide the protocol - it is simply not needed. There are many open protocols being used, and it would be easy to detect those. Keys can usually also be detected, even if meta information such as above is not available. If ECDH is used then the participants need to exchange public keys. It is usually possible to detect the keys from the handshake messages, e.g. by checking that the keys are specifically on a curve.

That all said, it is likely possible to create a protocol which is hard to reverse engineer if you just get the bytes from intercepted messages. But remember: even if the protocol is kept secret, it only requires a single leak to find what protocol is being used. As such, the hiding the protocol itself can be thought of as an obfuscation effort; it provides only limited security (this is usually referred to as the Kerckhoff principle).

Conclusion: sometimes it might be tricky to find out how a protocol is behaving when looking at the actual bits and bytes. However, generally the protocol descriptions are readily available and the security is determined by the keys and algorithms. Even if the protocol description is not directly available it is possible to figure out a lot of detail by reverse-engineering the messages (in the handshake).

ng flag

If a man in the middle attacker did not know in advance what kind of key exchange you were going to use, would that attacker be able to infer how you are exchanging keys?

Yes, that's at least quite plausible.

Even a passive attacker (that is one who can listen to data, but can not modify) would often be able to guess what kind of key exchange is going on. For example, if they see each party sending a blob of 384 mostly random bytes, an educated guess is they are using Diffie-Hellman key exchange in some group modulo a prime $p$ of about 3072-bit, and that blob is an ephemeral public key $a$ with $0<a<p$.

A Man in the Middle (MitM) attacker can pinpoint things even more accurately. For example, they can send a value of $a$ to a party, and look if that's accepted (indicating $a<p$) or rejected ($a\ge p$), perhaps with an error indication or a different timing. That allows to find the exact value of $p$ by binary search with a few thousands connection attempts; or/and confirm a guess (e.g. that $p$ is this standard value) with just two attempts.

This could be generalized to ECDH, authenticated ECDH… That's very general: in cryptography the method used can often be guessed by observation (of data exchanged, or seized gear/software). That's why, in modern cryptography, methods are assumed public and known to adversaries. Only secret keys (in symmetric crypto) and private keys (in public key crypto) are assumed to be unknown to adversaries on the long term. That's known as Kerckhoffs's (second) principle, and it's the default rule in the field of cryptography-as-a-science (for exceptions, see this answer).


Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.