Now, as I understood so far, when one of the user create the chat, the server will go to generate 2 couples of public and private key, and the next step is to exchange the public keys between the users and save locally the private ones.
It's called a public and private key pair.
What kind of actions are performed depends entirely on the protocol.
Generally the private key is kept on the end entity device, and the public key is shared and trusted somehow on the other devices. Most chat protocols are designed in such a way that the server cannot read the messages, or pose as an end entity. That means that the private keys for the end entities are not shared with the server.
Actually, in the article you've linked it reads:
Secure private key distribution among a user’s multiple devices is not a good option.
Now the problem arise when the user log out of the application, or simply delete it, since the private key is not stored anywhere in the server, but only on the local memory, how can the user recover the chat?
Well done, you've spotted one of the main problems with encrypted chat applications.
Actually, it's a bit worse as you'd encounter the same problem when adding a device. One way of handling this is to store the private key somewhere, possibly password encrypted. It can then be imported into the device again.
That sounds fine until you would want to protect the private key into a secure key store which precludes import of the private key and - more importantly - extraction of the key. In that case you might need to create a scheme with multiple levels of keys, for instance.
In short: key management is the hard part of any cryptographic protocol.
