Score:0

Increase image pull timeout on Kubernetes

au flag

We have multiple Kubernetes clusters ranged across multiple locations all with varying internet speeds.

Sometimes some of the smaller images are not able to be pulled by Kubernetes, with Kubernetes going into a ImagePullBackoff. These images are able to be pulled fine when running docker pull <image>. Some of these images are also massive ranging in the 6-10GB range.

We are running Kubespray 2.20 which Installs Kubernetes 1.24.6 and run docker 20.10 as the container engine.

I have tried setting the request timeout on the API server to 10 minutes and even 30 minutes but it does not seem to adjust the image pull timeout. Searching around and finding various other responses for this issue and all seem to have flags that are now deprecated and removed from Kubernetes. We are just looking for a way to increase the timeout for pull images in Kubernetes.

Score:1
nl flag

Try changing the kubelet configuration, specifically by increasing the runtimeRequestTimeout setting documented at https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/

See below potential solutions:

1)Find a new global default (like increasing it to 10 minutes or similar, but this might have unintended consequences)

2)Make this number configurable by the user via the UI

3)Give users the ability to modify the kubelet config file on disk and change k8s to use this file when it exists

Also go through how to Make kubelet flag --image-pull-progress-deadline configurable for more information may help to resolve your issue.

Matthew Moore avatar
au flag
This was the fix, for my use case I also had to overwrite kubespray's version of cri-dockerd to 0.2.6 as this had a fix that allowed the flag to actually be used.
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.