Ceph alertmanager config

Froxz

11/10/23, 7:54 AM

I've installed ceph using cephadm

Including monitoring stack prometheus, alertmanager, node-exporter

Currently I'm trying to add a telegram receiver (Telegram is supported from v0.24.0, so I've manually updated mgr/container_image_alertmanager fro 0.23 to 0.24) for alertmanager, but can't find in docs where alertmanager.yml should be created.

I can see this file is created inside ceph cluster /var/lib/ceph/{hash}/alertmanager.ceph-1/etc/alertmanager/alertmanager.yml

I've added the config to the above file as below:

templates:
  - '/etc/alertmanager/config/*.tmpl'
route:
  receiver: 'default'
  routes:
    - group_by: ['alertname']
      group_wait: 10s
      group_interval: 10s
      repeat_interval: 30m
      receiver: 'telegram'
receivers:
- name: 'default'
  webhook_configs:
- name: 'ceph-dashboard'
  webhook_configs:
  - url: 'https://ceph-1:8443/api/prometheus_receiver'
- name: 'telegram'
  telegram_configs:
    - bot_token: <bot_token>
      chat_id: <chat_id>
      send_resolved: true
      parse_mode: 'HTML'
      api_url: 'https://api.telegram.org'
      message: '{{ template "telegram.text" . }}'

The receiver works fine, but after redeploying the alertmanager from Ceph dashboard the config is gone, and is logical, becuase I'm editing generated file.

Please if anyone can assist and/or point to correct direction where I should create alertmanager config to extend/override the defaults.

490

1 + 0

ceph

prometheus

Score:1

Server

Ginnungagap

11/13/23, 7:53 AM

Reading the ceph manual regarding its monitoring, there's this part about using custom configuration for components of the stack.

Option names

The following templates for files that will be generated by cephadm can be overridden. These are the names to be used when storing with `ceph config-key set`:
 - services/alertmanager/alertmanager.yml
 - services/grafana/ceph-dashboard.yml
 - services/grafana/grafana.ini
 - services/prometheus/prometheus.yml
 - services/prometheus/alerting/custom_alerts.yml
 - services/loki.yml
 - services/promtail.yml

You can look up the file templates that are currently used by cephadm in src/pybind/mgr/cephadm/templates:
 - services/alertmanager/alertmanager.yml.j2
 - services/grafana/ceph-dashboard.yml.j2
 - services/grafana/grafana.ini.j2
 - services/prometheus/prometheus.yml.j2
 - services/loki.yml.j2
 - services/promtail.yml.j2

Simply take the ceph template, edit it to your liking and ceph config-key set it as the template to use when generating alert manager's configuration.

+ 0

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Ceph alertmanager config

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.