Score:0

Flannel network not getting configured correctly, how to fix that?

ke flag

I created an nginx deployment and got below error.

root@kmaster:~# kubectl describe pod nginx
Name:           nginx-6799fc88d8-ltnc5
Namespace:      default
Priority:       0
Node:           kworker1/10.8.0.70
Start Time:     Wed, 14 Jul 2021 16:26:32 +0000
Labels:         app=nginx
                pod-template-hash=6799fc88d8
Annotations:    <none>
Status:         Pending
IP:             
IPs:            <none>
Controlled By:  ReplicaSet/nginx-6799fc88d8
Containers:
  nginx:
    Container ID:   
    Image:          nginx
    Image ID:       
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-ngqhw (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  kube-api-access-ngqhw:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason                  Age                    From               Message
  ----     ------                  ----                   ----               -------
  Normal   Scheduled               7m27s                  default-scheduler  Successfully assigned default/nginx-6799fc88d8-ltnc5 to kworker1
  Warning  FailedCreatePodSandBox  7m27s                  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "bd465da8133b3b31cea3ccf75b0bbf718336f4a06fea40aeef5ddf9a2ba74b96": open /run/flannel/subnet.env: no such file or directory
  Warning  FailedCreatePodSandBox  7m16s                  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "eb61496041afad0a51a989e1fe1898db37a7356cd1047e0e9bad73d5bfcc9abc": open /run/flannel/subnet.env: no such file or directory
  Warning  FailedCreatePodSandBox  7m5s                   kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a56c7fe7d8b102f38c4a84d514e18f5b995210c8efa3e945f6f12d51fc27b72b": open /run/flannel/subnet.env: no such file or directory
  Warning  FailedCreatePodSandBox  6m54s                  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1ebe31cebdac61145dbacbca661c2595ca267f4112fe77f69e7e1cd1f04b97bf": open /run/flannel/subnet.env: no such file or directory
  Warning  FailedCreatePodSandBox  6m42s                  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "3de40514e0b210f27e5fe3e3c70afafef5418a23e1faed886358766d2454d434": open /run/flannel/subnet.env: no such file or directory
  Warning  FailedCreatePodSandBox  6m29s                  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "1fba1c9980010c57d1dbb8ed8c6554ca16957f3b1773874fa8d652442212f6ed": open /run/flannel/subnet.env: no such file or directory
  Warning  FailedCreatePodSandBox  6m17s                  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "f1296e109c11689f7f01a0c5cb8483b6d47269092dc0851afce21bb17d873dca": open /run/flannel/subnet.env: no such file or directory
  Warning  FailedCreatePodSandBox  6m6s                   kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "23bfe01e93b6639f4953ee8590576e4c3784a01372afcf609affbd03d5b26398": open /run/flannel/subnet.env: no such file or directory
  Warning  FailedCreatePodSandBox  5m51s                  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "2a589a2995a530c7d9b11abd51e7315852285078d05717aefa6c8f4c0e928866": open /run/flannel/subnet.env: no such file or directory
  Warning  FailedCreatePodSandBox  119s (x17 over 5m37s)  kubelet            (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "cd97f555abf734ef27d19ab7a13f1935dbf2e4fd2fca9e9cbb986061f174a46b": open /run/flannel/subnet.env: no such file or directory

I applied this to create flannel network Please suggest how to fix this?

lxc containers:

dminuser@testing:~/Desktop/kubernetes/lxd-provisioning$ lxc list
+----------+---------+-------------------+---------------------------------------------+-----------+-----------+
|   NAME   |  STATE  |       IPV4        |                     IPV6                      |   TYPE    | SNAPSHOTS |
+----------+---------+-------------------+---------------------------------------------+-----------+-----------+
| kmaster  | RUNNING | 10.8.0.129 (eth0) | fd42:666f:471d:3d53:216:3eff:fedb:8af1 (eth0) | CONTAINER | 0         |
+----------+---------+-------------------+---------------------------------------------+-----------+-----------+
| kworker1 | RUNNING | 10.8.0.229 (eth0) | fd42:666f:471d:3d53:216:3eff:fe93:ca5e (eth0) | CONTAINER | 0         |
+----------+---------+-------------------+---------------------------------------------+-----------+-----------+
| kworker2 | RUNNING | 10.8.0.152 (eth0) | fd42:666f:471d:3d53:216:3eff:fe23:b3a4 (eth0) | CONTAINER | 0         |
+----------+---------+-------------------+---------------------------------------------+-----------+-----------+

some output related to pods:

oot@kmaster:~# kubectl get nodes
NAME       STATUS   ROLES                  AGE     VERSION
kmaster    Ready    control-plane,master   3m44s   v1.21.0
kworker1   Ready    <none>                 114s    v1.21.0
kworker2   Ready    <none>                 99s     v1.21.0

oot@kmaster:~# kubectl get pods -n kube-system
NAME                              READY   STATUS              RESTARTS   AGE
coredns-558bd4d5db-569mm          0/1     ContainerCreating   0          4m1s
coredns-558bd4d5db-pzlfh          0/1     ContainerCreating   0          4m1s
etcd-kmaster                      1/1     Running             0          4m4s
kube-apiserver-kmaster            1/1     Running             0          4m4s
kube-controller-manager-kmaster   1/1     Running             0          4m4s
kube-flannel-ds-8mfsv             0/1     Error               2          2m14s
kube-flannel-ds-jv9rg             1/1     Running             4          4m1s
kube-flannel-ds-wms5n             0/1     CrashLoopBackOff    2          2m29s
kube-proxy-k8mnl                  0/1     CrashLoopBackOff    5          4m1s
kube-proxy-mw8ff                  0/1     CrashLoopBackOff    4          2m14s
kube-proxy-qkpzl                  0/1     CrashLoopBackOff    4          2m29s
kube-scheduler-kmaster            1/1     Running             0          4m4s
root@kmaster:~# 


oot@kmaster:~# kubectl logs kube-flannel-ds-8mfsv 
Error from server (NotFound): pods "kube-flannel-ds-8mfsv" not found
root@kmaster:~# kubectl logs kube-flannel-ds-8mfsv -n kube-system
I0715 08:13:32.125253       1 main.go:520] Determining IP address of default interface
I0715 08:13:32.125810       1 main.go:533] Using interface with name eth0 and address 10.8.0.152
I0715 08:13:32.125874       1 main.go:550] Defaulting external address to interface address (10.8.0.152)
W0715 08:13:32.125954       1 client_config.go:608] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
E0715 08:14:02.128488       1 main.go:251] Failed to create SubnetManager: error retrieving pod spec for 'kube-system/kube-flannel-ds-8mfsv': Get "https://10.96.0.1:443/api/v1/namespaces/kube-system/pods/kube-flannel-ds-8mfsv": dial tcp 10.96.0.1:443: i/o timeout
moonkotte avatar
in flag
I can see you've already created a couple of questions about `flannel`. To addition of what suggested, how did you set up your cluster? What pod's CIDR is used? And yes, it looks like flannel pod is crashing. Please check `kubectl get pod -n kube-system' and then `kubectl desribe pod %CRASHING_POD_NAME% -n kube system`. Please update the question with this output
uday avatar
ke flag
There is one more problem with lxc containers, after restart of the machine, It is not starting the kubelet process itself on master, so I will create the cluster again and reply with the output in few minutes. ALso, can you please reply to my istio related question also.
uday avatar
ke flag
added output for the commands you asked in the master
moonkotte avatar
in flag
Have you fixed your issue with lxc containers? This is the first thing you should start with.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.