It is possible to use ChaCha20 in a CMAC construction. Abstractly, we can view CMAC as a generic construction to build variable input length pseudo-random functions. It is well-known that a secure PRF is also a secure MAC. So technically, one could also directly use Chacha20 as a MAC by first applying a collision-resistance hash, producing outputs whose size is equal to the size of ChaCha20 inputs.
So, let's look somewhat closely at the CMAC construction. It is built out of (1) a variable-length prefix-free secure pseudo-random function and (2) a randomized prefix-free encoding (with high probability). Condition (1) means that we have a primitive that is a secure PRF as long as none of its possible inputs are prefixing each other. For 2), I'll just mention that this is achieved with the subkeys and XORing the last blocks in CMAC. How do we build such variable-length prefix-free PRF? A possibility that is also the one used in CMAC is the CBC mode of operation. Although usually used with a block cipher for decryption, CBC can also be used with a PRF (recalling that a block cipher can be used as a secure PRF).
As for security, there are no concerns since ChaCha20 is conjectured to be a PRF. You may get a better security bound than shown in this answer since AES-based CMAC introduces a factor due to the PRP-PRF switching lemma.
A full security argument for CMAC based on a PRF is presented in Boneh&Shoup's book (section 6.3).
The comparison between AES and ChaCha20 above is probably unfair since AES has a smaller output size, but for the same output-size an actually PRF is desirable.