Score:0

What is the right way to assign Network Contributor Role to an AKS cluster via ARM / Bicep template?

cn flag

I'm trying to configure a Load Balancer for my AKS server using Bicep/ARM. I am using the NGinx Ingress Controller in kubernetes and it does seem to work but when I first spin things up I am encountering an error.

Mainly I'm wondering what is the equivalent ARM or Bicep template for this step in the Azure documentation?

https://docs.microsoft.com/en-us/azure/aks/static-ip#create-a-service-using-the-static-ip-address

az role assignment create \
    --assignee <Client ID> \
    --role "Network Contributor" \
    --scope /subscriptions/<subscription id>/resourceGroups/<resource group name>

I'm using Bicep and have created my AKS server like this for example:

resource ExampleKubernetes 'Microsoft.ContainerService/managedClusters@2021-07-01' = {
  // ...
}

I'm then adding a role assignment to the kubelet identity like so:

var NetworkContibutor = '4d97b98b-1d4f-4787-a291-c67834d212e7'
resource AssignNetworkContributorToKubelet 'Microsoft.Authorization/roleAssignments@2020-08-01-preview' = {
  name: guid(resourceGroup().id, ExampleKubernetes.id, NetworkContibutor)
  dependsOn: [
    ExampleKubernetes
  ]
  properties: {
    roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', NetworkContibutor)
    principalType: 'ServicePrincipal'
    principalId: ExampleKubernetes.properties.identityProfile.kubeletidentity.objectId
  }
}

Which seems to work, I can see the Role assigned to the managed principal in the dashboard... but the Service in kubernetes seems to fail with a permission issue still:

  Error syncing load balancer: failed to ensure load balancer: Retriable: false,
  RetryAfter: 0s, HTTPStatusCode: 403, RawError: Retriable: false, RetryAfter:
  0s, HTTPStatusCode: 403, RawError:
  {"error":{"code":"AuthorizationFailed","message":"The client
  '<some guid A>' with object id
  '<some buid A>' does not have authorization to perform
  action 'Microsoft.Network/publicIPAddresses/read' over scope
  '/subscriptions/<subid>/resourceGroups/example/providers/Microsoft.Network/publicIPAddresses/example'
  or the scope is invalid. If access was recently granted, please refresh your
  credentials."}}

What's weird is that later on at some point it seems to just magically work. That error says "retriable false" and it does seem like the service doesn't retry but a subsequent deploy of NGinx to kubernetes will then cause it to retry and suddenly boom its working.

It just seems like the error message is telling me there is some non-deterministic delay of role propagation... So my questions are:

  • Is that right? Is it in fact just a delay and my code is basically right?
  • Am I using the right principalId? Or is that actually unnecessary?
  • Is there a way for me to force those role updates to propagate? I could have a CLI step in between if I needed to. How can I wait to install my ingress controller which connects to the LB after the permissions are ready?
Score:0
us flag

Your question (although not directly) is answered here.

The behavior you are describing is discussed in this section. Since Azure Resource Manager sometimes caches configurations and data to improve performance, it can sometimes take up to 30 minutes for changes to take effect when you assign roles or remove role assignments.

Using Azure CLI, you can force a refresh of your role assignment changes by signing out and signing in.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.