Should you encrypt data between 2 servers in the cloud?
Yes.
Modern security thinking is that you don't consider your own network / datacenter as more trusted (than your WAN or the regular internet).
Traditionally one would allow for more relaxed security standards in the datacenter, within the "secure" perimeter of your own network. Both internal systems and users would be trusted, implicitly expected to be secure and never abusive or malicious. One only added for example TLS for connections crossing the perimeter and borders of your "secure" internal network.
Nowadays the increasingly more prevalent security concept is one of "zero trust", which abandons the concept of a secure and trusted internal networks/systems/users and applies the same rigorous level of security everywhere, regardless.
So for two back-end servers exchanging information with each-other:
- both servers and and all their services should be configured with TLS certificates (for server authentication and transport encryption)
- their communication should be encrypted
- clients should authenticate to services (with username password, a token, client certificate or whatever is suitable)
- your applications/(micro-)services should still do input validation and not trust the input from the internal clients/backend-systems to always be correct and safe to use verbatim.
- etc.
- etc.
In response to your edit
I just wanted to know who will be able to see that traffic between those servers
(server A is on AWS us-east for example and server B is in
Azure)
Unless Amazon and Microsoft have their own physical datacenter interlinks, traffic between AWS and Azure clouds will be routed over the public internet and/or transit one or more network segments operated by third parties. The exact path your traffic takes and which third parties that are can change at any moment due to how routing protocols and the internet work.
When you don't set up transport encryption that traffic will be in clear text and anybody with access to any segment can trivially eavesdrop.