I've create a Kubernetes Cluster with 3 worker nodes and am experiencing issues with Flannal in a constant CrashLoopBackoff:
kube-flannel kube-flannel-ds-7lc76 0/1 CrashLoopBackOff 7 (4m52s ago) 15m
kube-flannel kube-flannel-ds-j8rnz 0/1 CrashLoopBackOff 7 (4m29s ago) 15m
kube-flannel kube-flannel-ds-jg4bt 0/1 CrashLoopBackOff 7 (4m15s ago) 15m
kube-flannel kube-flannel-ds-vnnmt 0/1 CrashLoopBackOff 7 (5m1s ago) 15m
Upon checking the logs, there is mention of an issue with VXLAN, which doesn't really give much more away:
Defaulted container "kube-flannel" out of: kube-flannel, install-cni-plugin (init), install-cni (init)
I0728 14:30:25.837078 1 main.go:212] CLI flags config: {etcdEndpoints:http://127.0.0.1:4001,http://127.0.0.1:2379 etcdPrefix:/coreos.com/network etcdKeyfile: etcdCertfile: etcdCAFile: etcdUsername: etcdPassword: version:false kubeSubnetMgr:true kubeApiUrl: kubeAnnotationPrefix:flannel.alpha.coreos.com kubeConfigFile: iface:[] ifaceRegex:[] ipMasq:true ifaceCanReach: subnetFile:/run/flannel/subnet.env publicIP: publicIPv6: subnetLeaseRenewMargin:60 healthzIP:0.0.0.0 healthzPort:0 iptablesResyncSeconds:5 iptablesForwardRules:true netConfPath:/etc/kube-flannel/net-conf.json setNodeNetworkUnavailable:true useMultiClusterCidr:false}
W0728 14:30:25.837347 1 client_config.go:617] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
I0728 14:30:25.867779 1 kube.go:145] Waiting 10m0s for node controller to sync
I0728 14:30:25.867978 1 kube.go:489] Starting kube subnet manager
I0728 14:30:26.868868 1 kube.go:152] Node controller sync successful
I0728 14:30:26.869136 1 main.go:232] Created subnet manager: Kubernetes Subnet Manager - ts-k8s-master
I0728 14:30:26.869155 1 main.go:235] Installing signal handlers
I0728 14:30:26.869728 1 main.go:543] Found network config - Backend type: vxlan
I0728 14:30:26.869781 1 match.go:206] Determining IP address of default interface
I0728 14:30:26.870814 1 match.go:259] Using interface with name eth0 and address 10.0.0.100
I0728 14:30:26.870893 1 match.go:281] Defaulting external address to interface address (10.0.0.100)
I0728 14:30:26.871025 1 vxlan.go:141] VXLAN config: VNI=1 Port=0 GBP=false Learning=false DirectRouting=false
E0728 14:30:26.876691 1 main.go:335] Error registering network: operation not supported
I0728 14:30:26.876887 1 main.go:523] Stopping shutdownHandler...
The following error message is making me wonder if this in compatible. I'm running on an ARM64 system (4 x RaspPi4) but had followed instructions here.
Error registering network: operation not supported
My router gateway IP is 10.0.0.1 with a subnet of 255.0.0.0. The master node is on 10.0.0.100 with workers on 10.0.0.101+. I create the cluster on kubeadm init with the pod-network-cidr=10.244.0.0/16 as instructed.
I did however receive an issue with kube-proxy which couldn't locate the Flannel config (/run/flannel/subnet.env), which probably points to a deeper issue (perhaps with the daemonset?), so manually created the file with the following content:
FLANNEL_NETWORK=10.244.0.0/16
FLANNEL_SUBNET=10.244.0.1/24
FLANNEL_MTU=1450
FLANNEL_IPMASQ=true
I'm at a loss as to how to proceed, as this seems to be running deeper within Flannel.
It may be useful to know that coredns does have pods from the pod network\Flannel range:
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-5d78c9869d-7vhqh 1/1 Running 1 (105m ago) 4h32m 10.244.0.2 ts-k8s-master <none> <none>
coredns-5d78c9869d-x8r79 1/1 Running 1 (105m ago) 4h32m 10.244.0.3 ts-k8s-master <none> <none>
