It's a typical case of NAT hairpinning. When the client (VM1 or even VM2) is on the same LAN as the server (VM2), the flow becomes asymmetrical: the reply will bypass the part doing NAT, so replies are not un-DNAT-ed and the client receives a reply from an address (VM2) it didn't ask anything to (instead of public Host) and is rejected.
The easiest way to overcome this is to do SNAT in addition to DNAT when this case happens. So in addition to:
1908K 138M MASQUERADE all -- * enp0s31f6 10.10.10.0/24 0.0.0.0/0
Note that for this case the outgoing interface is not the public-facing interface but the interface toward VM2 (it's vmbr0).
One could add (here with a specific filter to port 443, this filter can be removed if the outcome is intended whatever the service):
iptables -t nat -A POSTROUTING -s 10.10.10.0/24 -d 123.123.123.123 -p tcp --dport 443 -j MASQUERADE
UPDATE: above rule doesn't work because the (initial) packet's destination 123.123.123.123 got changed to 10.10.10.3 at nat/PREROUTING. The one below would work instead (it queries Netfilter about the initial destination before DNAT happened):
iptables -t nat -A POSTROUTING -s 10.10.10.0/24 -m conntrack --ctorigdst 123.123.123.123 -p tcp --dport 443 -j MASQUERADE
but considering this is the least useful example and needlessly complex, let's just forget it.
Instead of above, especially if the host were to have multiple IP public IP addresses and/or multiple internal LANs (each with a client and a server in the same place), it's more useful to query Netfilter to know when this has to be done: when DNAT already happened first (and then it doesn't even require to specify a port to handle all cases):
iptables -t nat -A POSTROUTING -m conntrack --ctstate DNAT -j MASQUERADE
Instead of MASQUERADE, one can choose any arbitrary source NAT to a network prefix (iptables' NETMAP) that matches the server's default route, so replies are correctly going through the host and correctly un-DNAT-ed. This allows to have logs on the server that can also be used to know which specific internal client did the query. As it's useful per-LAN, the LAN should be stated again and the rule used multiple times for multiple LANs (which is not OP's current case, but this answer aims to consider more cases).
For example if, not being used anywhere else, the chosen fictive network is 10.11.10.0/24 (which will be part of the default route of VM2), instead of above use this:
iptables -t nat -A POSTROUTING -s 10.10.10.0/24 -m conntrack --ctstate DNAT -j NETMAP --to 10.11.10.0/24
VM2's logs can now show 3 cases:
query from public IP address: actual client address, including the host if it chose to query to 123.123.123.123 (with the socket initially choosing from 123.123.123.123 so this is the source) before being redirected by nat/OUTPUT.
query from an address in 10.10.10.0/24: direct query from LAN (including a query from the host) to VM2 without using 123.123.123.123 as destination.
query from an address in 10.11.10.0/24: query from LAN to 123.123.123.123 which was redirected. VM1 will always appear as 10.11.10.2 source because NETMAP keeps the host part intact.
This last part will also allow to distinguish if VM1 did the query, or say a new VM VM3 initially at 10.10.10.4 seen as 10.11.10.4.
