How to expose services in private AKS cluster?

Sentrigan

3/9/23, 3:22 PM

I've created a private cluster on AKS and deployed some workloads to it, but I'm not sure how to connect to the services. They are all NodePort services, both TCP & UDP.

Initially, I thought that the endpoint AKS provides could be used to interact with the cluster as a whole, but this isn't the case -- this only exposes the Kubernetes API (probably why they call it the API endpoint)

I also tried using the VMSS public IP but that didn't work either. Using the instance IP works, but the IPs would change over time, right?

Ultimately I decided to use an ingress controller and proxy the traffic, but because I'm using mixed protocols I would need two. Not to mention this seems too complex to be the reasonable solution to this problem.

How to expose services on a private AKS cluster? This should be a pretty common problem, I think, but I haven't found a clear solution

130

0 + 0

networking

azure

kubernetes

aks

Score:1

Server

Sentrigan

3/11/23, 7:08 PM

If anyone ever finds themselves in this situation, we ended up using ExternalDNS and a private zone, since we're using NodePorts anyways.

This gives us a stable endpoint which will be automatically updated if one of the node's IPs ever changes.

Something to note is that the documentation implies no role assignment over the subscription is needed (By default, create-for-rbac grants Contributor on the subscription to the service principal which --skip-assignment avoids) but this didn't work for me, ExternalDNS got permission denied when trying to read the private zones.

I tried some different combinations but ended up giving Private DNS Zone Contributor on the subscription and skipping any assignments to the zone and resource group.

Also want to point out a gotcha, there are two providers for Azure, azure for public DNS and azure-private-dns for private DNS, and they differ slightly in configuration.

0 + 0

Paul

3/15/23, 4:58 PM

Welcome to Server Fault! Thank you for sharing the solution you discovered for your question. Please accept your answer to prevent your question from cycling through SE-enforced merry-go-round of active questions.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: How to expose services in private AKS cluster?

TH: จะเปิดเผยบริการในคลัสเตอร์ AKS ส่วนตัวได้อย่างไร

RO: Cum să expun serviciile în clusterul privat AKS?

RU: Как предоставить службы в частном кластере AKS?

VI: Làm cách nào để hiển thị các dịch vụ trong cụm AKS riêng?

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.