How do I configure Kubernetes to use kubenet with kubeadm?

I deployed a Kubernetes cluster using kubeadm.

$ kubectl get nodes
NAME                                          STATUS     ROLES                  AGE   VERSION
ip-172-31-33-9.us-east-2.compute.internal     NotReady   <none>                 48m   v1.22.2
ip-172-31-46-53.us-east-2.compute.internal    NotReady   control-plane,master   49m   v1.22.2
ip-172-31-47-245.us-east-2.compute.internal   NotReady   <none>                 6s    v1.22.2

The nodes are NotReady because I haven't installed a networking plugin. I want to use keep it simple and use kubenet. The instructions say:

Kubelet must be run with the --network-plugin=kubenet argument to enable the plugin

but I can't figure out how to do that. I have tried adding it to various configuration files, such as /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf and /etc/sysconfig/kubelet, and restaring kubelet, but none of it works.

How do I use kubenet with kubeadm?

You can use the /etc/default/kubelet file to override kubelet arguments. As can be found in the 10-kubeadm.conf file:

...
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use
# the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/default/kubelet
...

I'll show you how it works.

First, I set up the Kubernetes control plane:

# kubeadm init
[init] Using Kubernetes version: v1.22.2
...

# mkdir -p $HOME/.kube
# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
# sudo chown $(id -u):$(id -g) $HOME/.kube/config

After successful initialization, we can check the value of the --network-plugin argument that kubelet is currently using and the status of the coredns Pods:

# ps aux | grep "kubelet" | grep "network-plugin"
root     27488  5.1  3.6 1816612 145808 ?      Ssl  10:42   0:01 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --network-plugin=cni --pod-infra-container-image=k8s.gcr.io/pause:3.5

# kubectl get pod -n kube-system | grep "coredns"
coredns-78fcd69978-bbc52          0/1     Pending   0          2m26s
coredns-78fcd69978-fdcv9          0/1     Pending   0          2m26s

From the above output, we can see that --network-plugin=cni and coredns Pods are in the Pending state.

Let's create the /etc/default/kubelet file and write the kubelet arguments there:
NOTE: You may need to customize the --pod-cidr and --pod-infra-container-image to suit your needs.

# touch /etc/default/kubelet

# echo 'KUBELET_KUBEADM_ARGS="--network-plugin=kubenet --pod-cidr=10.20.0.0/24 --pod-infra-container-image=k8s.gcr.io/pause:3.5"' > /etc/default/kubelet

# cat /etc/default/kubelet
KUBELET_KUBEADM_ARGS="--network-plugin=kubenet --pod-cidr=10.20.0.0/24 --pod-infra-container-image=k8s.gcr.io/pause:3.5"

Then we need to reload systemd manager configuration and restart kubelet:

# systemctl daemon-reload
# systemctl restart kubelet

Finally, we can check if it works as expected:

# ps aux | grep "kubelet" | grep "network-plugin"
root     27841  6.9  3.5 1890600 143760 ?      Ssl  10:47   0:01 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --network-plugin=kubenet --pod-cidr=10.20.0.0/24 --pod-infra-container-image=k8s.gcr.io/pause:3.5
   
# kubectl get pod -n kube-system | grep "coredns"
coredns-78fcd69978-bbc52          1/1     Running   0          6m51s
coredns-78fcd69978-fdcv9          1/1     Running   0          6m51s   

# kubectl get nodes
NAME      STATUS   ROLES                  AGE   VERSION
kmaster   Ready    control-plane,master   15m   v1.22.2

You can follow these steps for all of your nodes.

Additionally, please remember that (more information can be found in the kubenet documentation):

Kubenet is a very basic, simple network plugin, on Linux only. It does not, of itself, implement more advanced features like cross-node networking or network policy. It is typically used together with a cloud provider that sets up routing rules for communication between nodes, or in single-node environments.

kubeadm gives the possibility to use a --config yaml file that is documented here. In this file you can use the cli parameters from kubelet as extra args like --network-plugin=kubelet as network-plugin: kubelet:

apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
bootstrapTokens:
...
nodeRegistration:
...
  kubeletExtraArgs:
    ...
    network-plugin: kubenet
    pod-cidr: 10.20.0.0/24