If the pre-shared secret is used only once, then it cannot be decrypted (and we don't even need to assume the message is random). So this would be a one-time pad and is information-theoretically secure even against an adversary with infinite computing power.
If, however, we do this again with another random message and the same key, an attacker can trivially XOR the two cipher texts and get the XOR of the two plain texts. This is already serious leakage, but if the two plain texts were chosen randomly, recovering them could be a challenge.
When the plain texts are not random, several techniques attack such a two-time pad, the leading of which is crib dragging.
For random plain texts, the question is: what will you do with them next? Although just sending random data in itself isn't terribly useful, if it's later used again, this may create an attack vector where knowing the XOR of two plain texts could be valuable.
In the comment, you suggest the plaintext will be used as the key for AES GCM, so it opens up a theoretical attack vector where we know two keys are related and probably will also have plaintext-ciphertext pairs for these related keys. I'm unaware of a practical related-key attack on full-length AES, but it definitely weakens the cipher.
An active attacker could go one step further and, after the first such XOR-based key exchange, set the second key to be a key with an arbitrary relation to the first.
