Score:0

Re-init kubernetes cluster

bd flag

I have a multimaster k8s cluster. I had to delete some of the master nodes, and had executed a kubeadm delete m2 and the same for the third node (m3), so that I could have only one master and rejoin the others later. However, this somehow messed with the main node (m1), which now gives me these errors:

Jan 12 08:56:29 k8s-m1 kubelet[14734]: E0112 08:56:29.314499   14734
eviction_manager.go:256] "Eviction manager: failed to get summary
stats" err="failed to get node info: node \"k8s-m1\" not found"

Jan 12 08:53:15 k8s-m1 kubelet[14734]: E0112 08:53:15.552154   14734
kubelet.go:2448] "Error getting node" err="node \"k8s-m1\" not
found"

Jan 12 08:56:29 k8s-m1 kubelet[14734]: E0112 08:56:29.571175   14734
event.go:276] Unable to write event:
'&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""},
ObjectMeta:v1.ObjectMeta{Name:"k8s-m1.1739835c5d7370d9",
GenerateName:"", Namespace:"default", SelfLink:"", UID:"",
ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1,
time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>,
DeletionGracePeriodSeconds:(*int64)(nil),
Labels:map[string]string(nil), Annotations:map[string]string(nil),
OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil),
ManagedFields:[]v1.ManagedFieldsEntry(nil)},
InvolvedObject:v1.ObjectReference{Kind:"Node", Namespace:"",
Name:"k8s-m1", UID:"k8s-m1", APIVersion:"", ResourceVersion:"",
FieldPath:""}, Reason:"NodeAllocatableEnforced", Message:"Updated
Node Allocatable limit across pods",
Source:v1.EventSource{Component:"kubelet", Host:"k8s-m1"},
FirstTimestamp:time.Date(2023, time.January, 12, 8, 46, 9,
272926425, time.Local), LastTimestamp:time.Date(2023, time.January,
12, 8, 46, 9, 272926425, time.Local), Count:1, Type:"Normal",
EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC),
Series:(*v1.EventSeries)(nil), Action:"",
Related:(*v1.ObjectReference)(nil), ReportingController:"",
ReportingInstance:""}': 'Post
"https://10.10.40.30:6443/api/v1/namespaces/default/events":
EOF'(may retry after sleeping)

The IP address is of the load balancer.

Is there a way to relive this master node, so that I don't have to recreate the whole cluster?

Fariya Rahmat avatar
ve flag
Which version are you using? 1.10 is no longer supported by the kubeadm team.Please note that you'd have to upgrade from 1.10->1.11->...1.14 and you cannot upgrade directly.My recommendation is to just re-recreate the cluster at this point.
Fariya Rahmat avatar
ve flag
Hello @francisaugusto, if you think that my answer helped you, please consider accepting it (✔️). I'd really appreciate it. Have a great day, thanks!
bd flag
Hi @FariyaRahmat. The problem had nothing to do with the version.
Score:0
bd flag

As this was an HA setup, I had no choice but to recreate the cluster.

Score:0
ve flag

Here are some troubleshooting steps which you can follow :

  1. Try to remove var/lib/kubelet and re-install kubelet.

  2. Restarting the kubelet and docker service.

    sudo service docker restart

    sudo systemctl restart kubelet

You can also refer to this document on Troubleshooting kubeadm.

I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.