Score:1

Why does my prometheus dashboard only show 2 weeks of data when I have longer term data available?

pt flag

I have Prometheus configured with Thanos. I'm using the Node Exporter Full dashboard to view data from the node exporter. Regardless of what time range I select (e.g., now-20d), the dashboard only shows 2 weeks of data:

enter image description here

If I go to the "Explore" UI instead and entry a query (such as node_memory_MemFree_bytes{instance="172.31.64.1:9100", job="node"}, which matches the data shown in the previous image), using the same data source and time range, I obviously have data going back much farther:

enter image description here

What is preventing the dashboard from showing data beyond the last two weeks?

Score:-1
ug flag

The default Prometheus retention time is 15 days.

From https://prometheus.io/docs/prometheus/latest/storage/#operational-aspects:

Prometheus has several flags that configure local storage. The most important are:
...
--storage.tsdb.retention.time: When to remove old data. Defaults to 15d. Overrides storage.tsdb.retention if this flag is set to anything other than default.

Retention can also be configured based on size:

--storage.tsdb.retention.size: The maximum number of bytes of storage blocks to retain. The oldest data will be removed first. Defaults to 0 or disabled. Units supported: B, KB, MB, GB, TB, PB, EB. Ex: "512MB"...

You can view your current retention setting on a running Prometheus instance by going to http://<IP_address_of_instance>:9090/status and looking for the "Storage retention" line.

pt flag
The data is coming from Thanos, not from Prometheus, and as noted in the question Thanos demonstrably has more than two weeks of retained data.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.