You can always encrypt traffic between any systems which belong to a defined group. IPsec transport mode is created specifically for that. It is not important which roles those servers assume, backend, frontend, etc., they are just IP nodes in this case. Consider this as a generic solution which makes "yes, it is possible" a valid answer for all questions like "is that possible to encrypt traffic between A and B". However, sometimes it is not convenient and so often there are other options.
Other options depend on for which purpose you need this encryption. Don't answer "just for security", there is no such thing as "just security". There are threat models and there are security models that cope with those defined threats. For example, in simple terms the threat model for the HTTP is the man in the middle who can eavedrop and inject its own data, impersonate itself as a valid server and/or a valid client; the HTTPS is designed to make this impossible by encrypting and signing all communication. MitM at best can either pass all the traffic without being able to eavedrop or just disrupt communication altogether. So, what you are defending against, what are your threats?
Your network between the backends and balancers is not trusted? Why? Those networks should include only balancers and backends and nothing else, which untrusted actors do you have there? However, IPsec in transport mode is acceptable solution in this case, because it will encrypt everything on the wire.
You can also use HTTPS between balancer and backends (and between backends themselves). That's OK, your balancer will terminate user HTTPS, see the request and reply in plain text, able to mangle them (add/remove/change headers), and able to select the backend and processing by analyzing contents, for example, select one backend for static and other for dynamic content. Then it will establish another HTTPS connection to backend. The only HTTPS clients for backends will be balancers and other backends, so it is not important for them to use globally recongized certificates. (For example, Google Frontends don't check certificates of HTTPS backends located inside Google Cloud, so even self-signed certificates will work.) You may create your own internal CA, issue certificates for all backends and balancers and make them trusted by installing that CA certificate in each system's store. Only the balancers need to have configured globally valid certificates and only on the client-facing side. Let's Encrypt automation will likely to be also the duty of those balancers, because issued certificates are to be installed on them.
You don't trust your load balancer? SSL passthrough could help, but it has its own drawbacks too. In this setup the balancer is one of those men in the middle against whom the HTTPS was created. So it can't access the HTTP details to balance requests upon, even know the http Host header value to differentiate vhosts; it can't inject additional proxy headers like those "proxied for", and so on; connections are protected from it as expected. All it could do is to hopefully track the connection and otherwise blindly forward packets to some backend. In this case you don't configure any certificates on it and even don't configure any server names. Just specify IPs of backends. Also, in this case valid certificates must be installed on all backends, because it is not known a priori which backend will get which request.
