I found a way to do it, but even though eventually it turned out to be not that difficult, still it requires recompiling some binaries so it's rather tough so far (but like I said, doable)
Hopefully the CNI plugins (part of k8s installation, see below) will be updated, then I will update my answer too. Meanwhile, here is a solution that so far worked perfectly for me:
This worked for vanilla kubeadm-based kubernetes installation, and probably would work for any kubernetes distribution/installation that allows to replace CNI plugins binaries. E.g. for a kubeadm-based way, if containerd is used, there is the following step:
...
Use the following commands to install Containerd on your system:
Follow the instructions for getting started with containerd...
In turn, the "getting started page" contains instructions to install the CNI plugins:
...
Step 3: Installing CNI plugins
... and extract it under /opt/cni/bin
...
... ./loopback
...
That's where the loopback plugin binary can be replaced (it can be done after k8s installation too as long as it's found on all nodes). The replacement one can be prepared using either the PR or the source fork repo (see multicast-on-loopback branch in the latter).
The actual change is rather simple, all that needs to be done is figuring out how to recompile the plugins after applying it (this also looks quite straightforward) and then replace the original loopback binary (found as described above) with the built one.
Notice that the binary needs to be replaced on all nodes, even though originally it's installed onto master only.
After the binary is replaced, just in case I restarted all nodes in the cluster, and voila - a perfectly workable multicast on loopback is here in each of the pods. Of course, the original loopback binary should be backed up first.
I checked that MC is isolated within a pod, to do that I've run iperf on multiple pods. From what I see it's all good, and it's actually isolated properly, i.e. loopback is still a loopback for a pod only.
