ChaCha20 is a stream cipher and outputs a stream $O_i$ to encrypt the messages to get the ciphertext $$C_i = M_i\oplus O_i$$ Your next encryption will be $$C_i' = M_i\oplus O_i \oplus O_i'$$ where the $O_i'$ is the output of the second ChaCha20 with a different key.
Well, what you are asking is the ciphertext-only attack on ChaCha20. No one is able to do this for even single encryption, yet there is no distinguisher. The output of a single ChaCha20 is good enough so that one cannot distinguish it from random. So, the attacker can see all possible messages as long as the message space is not short or some other verification methods exist like a server acts as an Oracle.
Since they have now retrieved the result of the second to last encryption (and not the original material), then how would they know that they had succeeded?
Removing layers is not helpful here since $$random \oplus message = random$$ So, there will be no information to verify.
This is not your actual problem, your actual problem will be the management of the keys. Be careful how you generate/derive them and how you store/derive.
Using a single ChaCha20 with 256 bits of the key is enough to be safe from all adversaries (classical or quantum) with a uniform random key. If you really want to use double encryption combine it with a different encryption algorithm like AES-GCM.
Keep in mind that, the multiple encryption (cascading) consideration and analyses started with the DES weakness consideration. Ueli M. Maurer & James L. Massey in 1993 showed in Cascade Ciphers: The Importance of Being First this;
- Cascade construction is secure as the first cipher and more security is not guaranteed. There is a blog post of Matthew Green on this subject; Multiple encryptions as noted by SAI Peregrinus.
For stream ciphers we have a different result ( again Maurer and Massey);
- Corollary 2. The bitwise modulo $2$ sum of $n$ keystream sequences that are generated by devices with independent keys is at least as difficult to predict as the most-difficult to-predict keystream sequence.