I'm trying to backfill Prometheus with old data. My input data is:
$ cat /etc/prometheus/backfill.txt
# HELP mymetric My metric.
# TYPE mymetric counter
mymetric{code="200",service="user"} 123 1665824331000
mymetric{code="500",service="user"} 456 1665824331000
# EOF
Following the docs, I then run the command which doesn't output any error:
$ promtool tsdb create-blocks-from openmetrics /etc/prometheus/backfill.txt /tmp/backfill_data
BLOCK ULID MIN TIME MAX TIME DURATION NUM SAMPLES NUM CHUNKS NUM SERIES SIZE
01GFNDV65PR5Z6MS5J82C6SAH4 1665824331000000 1665824331000001 1ms 2 2 2 787
And then copy the created folders to Prometheus' data folder:
$ cp -r /tmp/backfill_data/* /prometheus/data
But then nothing happens: this metric is not listed in Prometheus. It seems however to be correctly stored in TSDB:
$ promtool tsdb analyze /prometheus/data
Block ID: 01GFNDV65PR5Z6MS5J82C6SAH4
Duration: 1ms
Series: 2
Label names: 3
Postings (unique label pairs): 4
Postings entries (total label pairs): 6
Label pairs most involved in churning:
2 service=user
2 __name__=mymetric
1 code=500
1 code=200
Label names most involved in churning:
2 __name__
2 code
2 service
Most common label pairs:
2 __name__=mymetric
2 service=user
1 code=200
1 code=500
Label names with highest cumulative label value length:
8 __name__
6 code
4 service
Highest cardinality labels:
2 code
1 __name__
1 service
Highest cardinality metric names:
2 mymetric
I tried to restart my Prometheus container but that didn't change anything.
Edit: I start Prometheus using the command arguments --no-scrape.adjust-timestamps --log.level=debug --config.file=/etc/prometheus/prometheus.yml --storage.tsdb.retention.time=2y --web.console.libraries=/usr/share/prometheus/console_libraries --web.console.templates=/usr/share/prometheus/consoles --web.config.file=/etc/prometheus/web.yml