Join Log in
Score:1
korolev avatar Crypto

In general, how do e2e messaging platforms deal with changes to the structure of the encrypted data?

tz flag
korolev

I'm a layman trying to deepen my understanding of crypto and private messaging by building a (proof-of-concept) centralized, end-to-end encrypted messaging "platform".

Messages are sent by devices to the "server" (running on my Pi inside my home network) where they are stored and from whence they can be retrieved by the recipients via an api call. The server knows just the bare minimum to distribute the messages correctly while the message text is encrypted and the whole message is signed properly.

I am now faced with the following challenge: I want to rename a property of the message format used by the client applications (think JSON: message.body → message.content). The problem now is, that (1) all the previous messages are stored on the server in the old format and (2) an "old" client may try to message a "new" client which forces me to somehow resolve this mismatch between the communicated messages' formats.

If the private keys were available to the server, this would be a trivial problem: simply transform the message to the recipient's expected message format. But since I'm trying to be truly private and the private keys are only know to their respective owners this isn't an option. The messages cannot be modified by the server.

I feel that at the heart of this problem there lies a more fundamental challenge which I'm sure people much more capable that I have come up with solutions and for which there exist standardized approaches. Thus here's my question: In general, how do e2ee messaging solutions (e.g. signal, matrix) handle changes to the structure of encrypted data? What approaches do exist except cutting off old versions and throwing away any message history?

I beg your pardon if this question sounds vague or is too broad. I'm just a noob trying to get the big picture.

Thanks a lot.

113
0 + 0
Maarten Bodewes avatar
in flag
Maarten Bodewes
Parsing or decryption isn't all that different: both need specific code. Generally resolved by putting a version number in the plaintext part of the message (and, preferably, including that in the signature calculation and/or AAD information of an authenticated cipher). If you don't have a version number now you need to shoe-horn it in. Take a look at other E2E transport security protocols; they will generally have such a version number.
1
Reply
kr flag
mentallurg
This question is off topic on Crypto SE. This question is actually about software design which supports multiple API version. It should be migrated to SO or to Software Engineering SE.
0
Reply
Score:3
fgrieu avatar Crypto
ng flag
fgrieu

This is the generic IT problem of backward compatibility, made harder by crypto.

In IT, a common way to deal with backward compatibility is having a version field at start of data structures, allowing applications to decide if they recognize the format, and handle it if they can. New applications handle old formats, at least during a transition period. This can be extended to cryptographic payloads by having the version field in clear, and the rest encrypted/authenticated according to the version field.

When the method to encrypt/authenticate does not change, there is no crypto-specific issue, and usual mechanisms to deal with enriching data structures without compatibility break will do (e.g. in JSON, it's possible to add new fields that old applciations will skip). In the following I assume the crypto has changed, and a version field allows to detect that.

The problem now is, that (1) all the previous messages are stored on the server in the old format and (2) an "old" client may try to message a "new" client which forces me to somehow resolve this mismatch between the communicated messages' formats.

If the new client can still decipher the old format, neither is an issue. That's the way to go!

The real hard problem is when a "new" client sends a message. There are many choices, neither of which is perfect. Some of these:

  • New clients always use the new format, which old clients won't understand.
  • New clients use the old format until some trigger (switch date, message from server), then use the new format. Old clients won't be usable after the switch, and new clients won't enjoy the new features of the new format until that event.
  • New clients use new format after the trigger or if there is some clue the receiver (or all the receivers) understands it, e.g. because the sending client has received an authenticated message from (all) the recipient(s) that indicates support of the new format, perhaps implicitly by using it.
  • New clients generate both formats until the trigger and send both to the server, which stores them; old clients fetch the old format, new clients fetch the new and in case of failure fetch the old format. The server must handle requests in both formats, and needs more storage and more traffic in the interim period.
0 + 0
korolev avatar
tz flag
korolev
Very clear answer. Thank you very much.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.