Score:0

microk8s: pod resource usage metrics not available from all nodes

in flag

I am running microk8s v1.22/stable on a Linux cluster with 11 nodes. I have enabled the metrics-server plugin and installed Prometheus via the Helm chart with nodeExporter and kubeStateMetrics enabled. I get all sort of other metrics, but I have noticed that pod metrics (e.g., container_cpu_usage_seconds_total) are available for pods running on just one of the nodes. I didn't notice anything special about this node.

I have tried disabling and re-enabling the metrics-server plugin and added spec.template.spec.hostNetwork: true to the metrics-server deployment as suggested elsewhere, but to no avail.

Here's the result of running microk8s status:

$ microk8s status
microk8s is running
high-availability: yes
  datastore master nodes: ...
  datastore standby nodes: ...
addons:
  enabled:
    dns                  # CoreDNS
    ha-cluster           # Configure high availability on the current node
    ingress              # Ingress controller for external access
    metrics-server       # K8s Metrics Server for API access to service metrics
    storage              # Storage class; allocates storage from host directory
  disabled:
    ....

What could I be missing and what should I check?

Andrew Skorkin avatar
tr flag
Just to clarify. Prometheus works, right? Are the metrics available from all nodes, apart from some metrics related to pod information? Do you have pods on other nodes (not just the node where pod metrics are available)?
mhusaini avatar
in flag
Yes, Prometheus is working. I am watching several other metrics that are behaving normally. And yes, there are pods on other nodes as well.
Andrew Skorkin avatar
tr flag
Sorry, but it is still not clear for me: can you get metrics from other nodes? Any metrics, not related to pod information.
mhusaini avatar
in flag
Yes, I do get other kind of metrics from other nodes, just not pod information.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.