Join Log in
Score:0
FLBzh avatar Server

Get kubernetes pods to take on a specific identity / role / job once started

mx flag
FLBzh

What I am presenting here is a simplified version of my setup and the syntax is wrong.

I have a list of high level "agents" that I want to deploy as pods on a kubernetes cluster.

Some nodes in this cluster have special hardware, some don't, but all the pods should use the same container.

Could look like (again: syntax is wrong):

agent1 wlan
...
agent8 wlan
agent9 wlan rs232.1
agent10 wlan rs232.2

I setup a deployment that roughly looks like that (the syntax is wrong):

deployment (standard nodes)
  replicas: 8
---
deployment (rs232, terminal 1)
  replicas: 1
  nodeSelector:
    rs232=1
  env:
    rs232=1 (because nodeSelector can't be passed nicely afaik)
---
deployment (rs232, terminal 2)
  replicas: 1
  nodeSelector:
    rs232=2
  env:
    rs232=2

I would like each of these agents to start up and take on one identity from the list, while having the matching hardware of course.

So like a pod would talk to a role distribution service like that:

pod - Hi, I'm pod/container-xyz and I have env/label/annotation rs232=2
service - Hi, well then you are now agent10
pod - OK I'll do the work of agent10 then
(service - cool, I'll add agent10 as an alias to you in the cluster DNS) <- that will be my next question

Same thing for agents no with special hardware: the service gives them one agent role each.

I tried something with StatefulSet but that does not fit the bill (because of the various nodeSelector and they are super slow to start).

I started to think about a dedicated self-implemented server pod that would keep my agent list and lease them to matching hardware (a bit like dhcp server) but I'm pretty sure I am not the only one having this problem and there must be a solution out there. I looked quickly at Zookeeper but I don't think it is for me. I am probably not finding the solution because my vocabulary is not the correct one I guess.

Has anybody got a good idea ? Am I going in the right direction ?

68
0 + 0
kkopczak avatar
ng flag
kkopczak
Which version of Kubernetes did you use and how did you set up the cluster? Did you use bare metal installation or some cloud provider?
0
Reply
FLBzh avatar
mx flag
FLBzh
Hi @kkopczak, thanks for your question but how is it relevant to my problem ?
0
Reply
kkopczak avatar
ng flag
kkopczak
Such information will help for reproducing your problem.
0
Reply
Score:1
avatar Server
us flag
vaizki

I would probably implement this with a MetaController CompositeController and a Custom Resource which defines the agents. Coding is required though. It looks much more complicated than it is.

Basically you regularly receive sync hooks over HTTP describing the current state of resources and you respond with JSON to create/destroy them.

0 + 0
FLBzh avatar
mx flag
FLBzh
Yes ! Thanks a lot. There is even an example that is a good base to implement what we need: https://github.com/metacontroller/metacontroller/tree/master/examples/indexedjob
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.