Why is FHE nontrivial?

If I understand correctly (please let me know if any of the following is wrong), a fully homomorphic encryption scheme $\mathcal{E}$ is such that for any messages $x, y$, $$ \mathcal{E}(x + y) = \mathcal{E}(x) + \mathcal{E}(y) \\ \mathcal{E}(x y) = \mathcal{E}(x) \, \mathcal{E}(y), $$ i.e $\mathcal{E}$ is a ring homomorphism. Since the function $\mathcal{E}$ has to be invertible (hence injective), we have by some basic isomorphism theorems that $\mathcal{E}$ is actually a ring isomorphism between suitably chosen rings. If we assume that the domain and codomain are both $R$, we are looking for a ring $R$ with a suitably complicated automorphism group.

Why not just let $R$ be some finite field and choose $$ \mathcal{E}_{r}(x) = r^{-1} x r, $$ with $r \in R$? Alice could now send Bob $\mathcal{E}_{r}(x_1), \dots, \mathcal{E}_{r}(x_n) $ without revealing $r$. Bob could perform addition and multiplication return an encrypted result that Alice will be able to decrypt via $$ \mathcal{E}^{-1}_{r}(x) = r x r^{-1}. $$ Since FHE is an active research topic, it can't be this easy. I'm just not proficient enough in crypto to know why.

Since FHE is an active research topic, it can't be this easy.

Well, with FHE, it is generally assumed that it is a public key encryption scheme, that is, someone with the public key can encrypt things. However, that's not the main issue.

One problem is that FHE needs to be nondeterministic; in particular, if you're given the encryption of two messages $E(M_1)$ and $E(M_2)$, you shouldn't be able to determine whether $M_1 = M_2$. Because your method doesn't add any randomness, encrypting the same message twice will yield the same ciphertext, which is observable to the adversary.

Another problem with $rxr^{-1}$ (at least if you're using matrix multiplication) is that the operation is linear (in terms of the field the matrix is defined over). That is, if you use an $N \times N$ matrix, then you can recover everything with $N^2$ known plaintext/ciphertext pairs. Actually, there are likely to be better attacks (using fewer pairs), however just this one is sufficient to disqualify the scheme.

$\mathcal{E}$ is actually a ring isomorphism between suitably chosen rings.

Confusingly, this is actually not true. This is because practically all known FHE schemes have each ciphertext have a "noise level" associated with it. For example, let $\mathcal{C}$ be some (ring) of possible ciphertexts. Let $\mathcal{C}^{\epsilon}\subseteq\mathcal{C}$ be a subset of this ring for ciphertexts of "noise level" $\epsilon$.

Then, one roughly gets that

$$\mathsf{Add}: \mathcal{C}^{\epsilon_0}\times \mathcal{C}^{\epsilon_1}\to \mathcal{C}^{\epsilon_0 + \epsilon_1}$$

$$\mathsf{Mul}: \mathcal{C}^{\epsilon_0}\times \mathcal{C}^{\epsilon_1}\to \mathcal{C}^{f(\epsilon_0,\epsilon_1)}$$

Here, $f$ depends on the particular FHE scheme under consideration. Anyway, this "noise level" is important as decryption is only correct if the noise level is "small" (which is determined by the system parameters of the FHE scheme). Consequentially, the following two situations are not equivalent

1. doing some plaintext computation, and
2. doing the same computation homomorphically, then decrypting.

They may be equivalent (i.e. the FHE may be correct) if the noise level is low enough. But one can always specify computations (in terms of solely $\mathsf{Add}$, $\mathsf{Mul}$, or even only a single one of these algorithms) for which the noise level gets too large, and decryption would be incorrect.

Daniele Micciancio has discussed this nuance some in this talk, where he distinguishes between "Fully Homomorphic Encryption" and "Fully Composable Homomorphic Encryption", but I don't believe there is a corresponding paper (I haven't watched the talk in a while though).