Join Log in
Score:1
rvs avatar Server

Setting resource limits on kube-apiserver

vn flag
rvs

Resource Management for Pods and Containers describes how to set resource requets and limits for "regular" pods in Kubernetes. Is there a supported/reccomended way to set these limits for control plane components such as kube-apiserver?

Things I considered:

  • Modifying static manifests, e.g. in /etc/kubernetes/manifests/kube-apiserver.yaml. This could work but it will be overwritten by kubeadm during next upgrade.
  • Setting kube-reserved or system-reserved flags. This could work too however again - they are defined in just one ConfigMap (e.g. kubelet-config-1.21) and will be overwritten by kubeadm during node upgrade. The same limits will apply to control plane nodes and worker nodes and I don't want that.

I can overcome this with something like ansible but then ansible will be "figting" with kubeadm and I'd like to avoid that.

What problem am I trying to solve?

I have a small homelab kubernetes installation. I'd like to allow running regular pods on control plane node(s), but I'd like to be able to reserve some resources (primarily memory) for control plane components. I.e. I'd like to be able to set requests on things like kube-apiserver so that scheduler knows not to put any other pods (they will also have appropriate requests) in its place.

101
0 + 0
Mikołaj Głodziak avatar
id flag
Mikołaj Głodziak
How exactly did you set up your cluster? What does it mean "I have a small homelab kubernetes installation."? Did you use Minikube?
0
Reply
rvs avatar
vn flag
rvs
@MikołajGłodziak no, "regular" cluster on 3 nodes built by kubeadm.
0
Reply
Score:1
Mikołaj Głodziak avatar Server
id flag
Mikołaj Głodziak

Is there a supported/reccomended way to set these limits for control plane components such as kube-apiserver?

Yes, you can use kubeadm init with patches command line flag. Look at this github page. The documentation of this thing could be also interested. See also official documentation: Customizing the control plane with patches:

Kubeadm allows you to pass a directory with patch files to InitConfiguration and JoinConfiguration on individual nodes. These patches can be used as the last customization step before the control plane component manifests are written to disk.

You can pass this file to kubeadm init with --config <YOUR CONFIG YAML>:

apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
patches:
  directory: /home/user/somedir

Here's example how to set resources on kube-apiserver:

Create kube-apiserver.yaml file in some directory (e.g. /home/user/patches) with following contents:

spec:
  containers:
    - name: kube-apiserver
      resources:
        requests:
          memory: 512Mi
        limits:
          memory: 1024Mi

Then use --patches flag every time during node upgrade: use kubeadm upgrade node --patches /home/user/patches/ or kubeadm upgrade apply v1.22.4 --patches /home/user/patches/

The other option will be supply extra flags to control-plane components. For this, check this guide: Customizing the control plane with flags in ClusterConfiguration:

The kubeadm ClusterConfiguration object exposes a way for users to override the default flags passed to control plane components such as the APIServer, ControllerManager, Scheduler and Etcd. The components are defined using the following structures:

  • apiServer
  • controllerManager
  • scheduler
  • etcd
0 + 0
rvs avatar
vn flag
rvs
Thank you, the patches route works. I've updated your answer with a more specific example of how I did that. Now I can use ansible to maintain different files in patches directory (if needed) and remember to use --patches flag when doing init or upgrades.
1
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.