Score:0

kube-proxy does not run, however shows in output of "kubectl get pod" as normally running

cn flag

The kube-proxy runs as a daemonset, so I see that there are two pods in my 2 nodes cluster:

kube-proxy-4f246                           1/1     Running   0          14d
kube-proxy-7hzkf                           1/1     Running   2          5m43s

After doing kubectl describe pod kube-proxy-7hzkf -n kube-system I get the ID of the container.

kube-proxy:
Container ID:  docker://53f81d08adaa9b29b7ac8d985c40d08500e84e1be96732ae7a91744a735df885

Then I am checking at the node that this kube-proxy is running that kube-proxy is running as a process, also there is a containerd process related to that kube-proxy instance:

ubuntu@k8s-2:~$ ps aux|grep 53f81d08adaa9b29b7ac8d985c40d08500e84e1be
root     31328  0.0  0.2 113964  8276 ?        Sl   12:21   0:00 /usr/bin/containerd-shim-runc-v2 -namespace moby -id 53f81d08adaa9b29b7ac8d985c40d08500e84e1be96732ae7a91744a735df885 -address /run/containerd/containerd.sock

Also:

ubuntu@k8s-2:~$ ps aux|grep kube-proxy
root     31355  0.0  1.0 747380 40520 ?        Ssl  12:21   0:00 /usr/local/bin/kube-proxy --config=/var/lib/kube-proxy/config.conf --hostname-override=k8s-2

If the kube-proxy process dies, this is what happens below, the Kubernetes is restarting the failed kube-proxy.

kube-system   0s          Normal    Pulled                    pod/kube-proxy-7hzkf                   Container image "k8s.gcr.io/kube-proxy:v1.21.1" already present on machine
kube-system   0s          Normal    Created                   pod/kube-proxy-7hzkf                   Created container kube-proxy
kube-system   0s          Normal    Started                   pod/kube-proxy-7hzkf                   Started container kube-proxy
default       0s          Normal    Starting                  node/k8s-2                             Starting kube-proxy.

However if the containerd-shim-runc-v2 process dies, the kube-proxy as well goes down , but then nothing is restarted and the node stays without kube-proxy, although everything seems OK in Kubernetes , no event shows up and the output of kubectl get pod etc. , kubectl describe shows the kube-proxy running!

What happens here??

ubuntu@k8s-1:~$ kubectl get pod -n kube-system
NAME                                       READY   STATUS    RESTARTS   AGE
..
..
kube-proxy-4f246                           1/1     Running   0          14d
kube-proxy-7hzkf                           1/1     Running   3          15m
kube-scheduler-k8s-1                       1/1     Running   0          47h
..

ubuntu@k8s-2:~$ ps aux|grep kube-proxy
ubuntu   10879  0.0  0.0  14864  1048 pts/0    S+   12:36   0:00 grep --color=auto kube-proxy
ubuntu@k8s-2:~$

So the last one shows no kube-proxy running!

Andrew Skorkin avatar
tr flag
Hello @maiky Do you know if kube-proxy on k8s-2 worked fine previously? Have you made any changes to your infrastructure/kubernetes last time? Which version of kubernetes are you using?
Wytrzymały Wiktor avatar
it flag
Hello @maiky. Any updates?
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.