For a small production environment is it better to use only masters k8s or some mini k8s solutions?

pk flag

I have a scenario of a small air-gap production environment with only three Linux servers (CentOS or RHEL). I want to deploy a small k8s cluster on them. I have two approaches for now:

  1. Installing a pure k8s cluster with only master nodes and untainting them from NoSchedule to run all pods on them.
  2. Installing a mini cluster solution using k3s, k0s, or microk8s and configuring all nodes as master and workers

If I use the first approach (I know it's a bad practice) is it the correct way to run pods on masters?

If I want to use the second one who is the best and easiest to install in different air-gap environments and maintain them? (I used k8s and okd 3 in production but not them)

Lastly, what do you think is the best approach from those two, or are there better ones for my scenario?

Thanks in advance for the help

mc flag

The big detail that's going to affect what's right for production is how much information security matters to you.

Kubernetes lets you have mechanisms (ResourceQuota, LimitRange, Pod security admission, pod priority, …) that protect apps in one namespace from being affected by another namespace. If you already know how to set these up - or you know that you'll have to learn them - then putting the apps onto control plane nodes of a ”regular” Kubernetes cluster can be a good fit.

That focus on infosec is going to make it important that you take appropriate hardening measures, and these are easier to do when the Kubernetes cluster you're configuring starts off like vanilla upstream Kubernetes. You might even want to set up gVisor to provide additional restrictions on what the application containers can do. If that's a story that's important for your context, have a look at the CKS syllabus to learn about other topics to consider.

If information security isn't a big focus, and perhaps you're keen to get something up and running soon, I think k3s would be a better fit. k3s does more for you out of the box, and makes it easier to run a replicated control plane with good availability.


Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.