Score:0

k3s pods not deployed on correct node

cn flag

I have a k3s cluster with several nodes, let's say 3 nodes. On each node, I have several pods to deploy. It's a "static deployment", meaning that I want to deploy each pod on a specific node. I don't want to let k3s do it automatically.

For example:

Node_1:
    pod_a
    pod_b
    pod_c
Node_2:
    pod_d
    pod_e
    pod_f
Node_3:
    pod_h
    pod_i
    pod_g

To achieve this, I'm using labels. I tag each node with specific labels and then I specify the label in the yaml deployment file:

For example, something like that:

apiVersion: v1
kind: Node
metadata:
  name: node3
  labels:
    db: 'true'

And the yaml deployment files, for example for the db:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: db
  namespace: default
spec:
    spec:
      nodeSelector:
        db: 'true'

So, normally the pod db should be deployed on Node3. But sometimes, it's not the case. When I do a redeployment, the pod redeploy on another Node and I don't understand why, and how to move it back to the correct node.

This is quite annoying because for some critical pods, I have data mounted on the host and if the pod is not deployed correctly my application can fail.

How to make sure my pod always deploys correctly? And why sometimes it fails to deploy correctly? Thank you

in flag
Based solely upon your description, it sounds like you have labeled multiple Nodes with `db: 'true'`, but that aside, if "want to deploy each pod on a specific node. I don't want to let k3s do it automatically" then use `nodeName:` instead of the label matching system and it'll put a stop to that altogether
Andrew Skorkin avatar
tr flag
Agree with @mdaniel about the same labels for multiple Nodes. Can you add an output for your Nodes with current labels which you want to use for pod allocation? Or you can try to use labeling of Nodes with providing appropriate nodeSelector in pod specs.
iAmoric avatar
cn flag
I confirm each label is on one and only one node. But, I checked again my yml and I just noticed one pod did not have a nodeSelector fireld. I'll fix it and test the deployment again
Andrew Skorkin avatar
tr flag
Hi @iAmoric Have you any updates about your deployment testing?
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.