You should hash your secret. (In fact, you should not just hash it, but process it with a key derivation function such as HKDF that has been specifically designed for this purpose.) Doing so preserves the entropy of the ECDH shared secret better than simple truncation and makes the security of your system easier to analyze, as you can rely on the output of the KDF being computationally indistinguishable from a uniform random bitstring.
All that said, as I noted earlier in this answer to a related question, skipping the hashing and just truncating the shared secret is unlikely to be a disastrous mistake. In particular, removing a byte from the secret can only reduce its entropy by at most 8 bits.
As Daniel S notes, an Ed25519 public key has about 251 bits of entropy. Truncating it by one byte thus leaves you with at least 251 − 8 = 243 bits of entropy.
Is that enough? Honestly, I can't think of any purpose for which it wouldn't be. For all currently foreseeable practical purposes, even 128 bits of entropy is enough (yes, even against quantum computers). A 243 bit secret is $2^{243 - 128} = 2^{115}$ times harder to crack by brute force than that.
Ps. Of course the back-of-the-envelope security analysis above considers only resistance to brute force guessing attacks. To be thorough, we should also consider the possibility of structural weaknesses, such as the possibility that the rest of your cryptosystem might somehow be secure except when its secret key happens to be a truncated Ed25519 shared secret.
In this particular case, however, that concern can be fairly easily dismissed by noting that a substantial fraction of all 31-byte strings — at least one in 32, to be precise — must be the truncation of some 32-byte Ed25519 shared secret, so that, if your cryptosystem was particularly vulnerable when used with such a string as its key, it would also have an unacceptably high chance of being vulnerable even when used with a perfectly random 31-byte key.
That said, here you can already see an example of what I meant about the lack of a proper KDF, and its guarantee of effective uniformity, complicating the analysis of the system. Basically, a KDF acts as a domain separator in your system, providing a guarantee that — as long as the KDF itself isn't broken and as long as you feed it something with enough entropy — you can safely analyze the rest of your system based on the assumption that the KDF outputs are indistinguishable from uniformly random. Without a KDF to provide this barrier, you now have to expand your security analysis to consider more details about exactly where your key material comes from and how its entropy is distributed.
