Score:1

I get a Certificate Signing Request every 15 minutes (Kubernetes)

cn flag

I get a Certificate Signing Request every 15 minutes

❯ kubectl get csr -A --sort-by=.metadata.creationTimestamp

csr-r2dwb   3h      kubernetes.io/kubelet-serving   system:node:host-cluster-control-plane-2hhtt   <none>              Pending
csr-kcj2m   165m    kubernetes.io/kubelet-serving   system:node:host-cluster-control-plane-2hhtt   <none>              Pending
csr-h4k8j   149m    kubernetes.io/kubelet-serving   system:node:host-cluster-control-plane-2hhtt   <none>              Pending
csr-rp5k8   134m    kubernetes.io/kubelet-serving   system:node:host-cluster-control-plane-2hhtt   <none>              Pending
csr-dpx5w   118m    kubernetes.io/kubelet-serving   system:node:host-cluster-control-plane-2hhtt   <none>              Pending
csr-f5zlj   103m    kubernetes.io/kubelet-serving   system:node:host-cluster-control-plane-2hhtt   <none>              Pending
csr-vmjrk   87m     kubernetes.io/kubelet-serving   system:node:host-cluster-control-plane-2hhtt   <none>              Pending
csr-q6nz7   72m     kubernetes.io/kubelet-serving   system:node:host-cluster-control-plane-2hhtt   <none>              Pending
csr-hhnfx   57m     kubernetes.io/kubelet-serving   system:node:host-cluster-control-plane-2hhtt   <none>              Pending
csr-bq2dl   41m     kubernetes.io/kubelet-serving   system:node:host-cluster-control-plane-2hhtt   <none>              Pending
csr-9cgws   26m     kubernetes.io/kubelet-serving   system:node:host-cluster-control-plane-2hhtt   <none>              Pending
csr-xmz2k   10m     kubernetes.io/kubelet-serving   system:node:host-cluster-control-plane-2hhtt   <none>              Pending

I already approved a cert via kubectl certificate approve csr-..., but nevertheless I get new CSRs every 15 minutes.

How to fix this?

root@host-cluster-control-plane-2hhtt:~# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Sep 11, 2023 19:57 UTC   334d            ca                      no      
apiserver                  Sep 11, 2023 19:57 UTC   334d            ca                      no      
apiserver-etcd-client      Sep 11, 2023 19:57 UTC   334d            etcd-ca                 no      
apiserver-kubelet-client   Sep 11, 2023 19:57 UTC   334d            ca                      no      
controller-manager.conf    Sep 11, 2023 19:57 UTC   334d            ca                      no      
etcd-healthcheck-client    Sep 11, 2023 19:57 UTC   334d            etcd-ca                 no      
etcd-peer                  Sep 11, 2023 19:57 UTC   334d            etcd-ca                 no      
etcd-server                Sep 11, 2023 19:57 UTC   334d            etcd-ca                 no      
front-proxy-client         Sep 11, 2023 19:57 UTC   334d            front-proxy-ca          no      
scheduler.conf             Sep 11, 2023 19:57 UTC   334d            ca                      no      

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Sep 08, 2032 19:55 UTC   9y              no      
etcd-ca                 Sep 08, 2032 19:55 UTC   9y              no      
front-proxy-ca          Sep 08, 2032 19:55 UTC   9y              no      

File /var/lib/kubelet/config.yaml

apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
  anonymous:
    enabled: false
  webhook:
    cacheTTL: 0s
    enabled: true
  x509:
    clientCAFile: /etc/kubernetes/pki/ca.crt
authorization:
  mode: Webhook
  webhook:
    cacheAuthorizedTTL: 0s
    cacheUnauthorizedTTL: 0s
cgroupDriver: systemd
clusterDNS:
- 10.96.0.10
clusterDomain: cluster.local
cpuManagerReconcilePeriod: 0s
evictionPressureTransitionPeriod: 0s
fileCheckFrequency: 0s
healthzBindAddress: 127.0.0.1
healthzPort: 10248
httpCheckFrequency: 0s
imageMinimumGCAge: 0s
kind: KubeletConfiguration
logging:
  flushFrequency: 0
  options:
    json:
      infoBufferSize: "0"
  verbosity: 0
memorySwap: {}
nodeStatusReportFrequency: 0s
nodeStatusUpdateFrequency: 0s
resolvConf: /run/systemd/resolve/resolv.conf
rotateCertificates: true
runtimeRequestTimeout: 0s
shutdownGracePeriod: 0s
shutdownGracePeriodCriticalPods: 0s
staticPodPath: /etc/kubernetes/manifests
streamingConnectionIdleTimeout: 0s
syncFrequency: 0s
volumeStatsAggPeriod: 0s

4 hours after approving the cert, I get new CSRs again and again:

csr-xmz2k   21h     kubernetes.io/kubelet-serving   system:node:host-cluster-control-plane-2hhtt   <none>              Pending
csr-d564x   16h     kubernetes.io/kubelet-serving   system:node:host-
...
...
...
csr-nn9tz   28m     kubernetes.io/kubelet-serving   system:node:host-cluster-control-plane-2hhtt   <none>              Pending
csr-h9k7j   12m     kubernetes.io/kubelet-serving   system:node:host-cluster-control-plane-2hhtt   <none>              Pending
asktyagi avatar
in flag
can you add `kubeadm certs check-expiration` output to your question as well?
guettli avatar
cn flag
@asktyagi I added the output of `kubeadm certs check-expiration`
asktyagi avatar
in flag
Is your kubelet configured to communicate over tls? if yes could you share what parameters have you used for kubelet process or share complete `/var/lib/kubelet/config.yaml` file?
guettli avatar
cn flag
@asktyagi I added `/var/lib/kubelet/config.yaml` to the question.
Score:1
cn flag

I found the solution: I created the cluster via Cluster-API.

The management cluster which created the workload cluster was in minikube on my laptop.

The workload cluster worked fine, but my laptop was off, so that auto-approving of the management cluster was not done.

After starting the management cluster inside minikube, everything worked fine again.

I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.