Score:0

Prometheus alert rule test fails with long intervals

mg flag

I've got an alert rule:

groups:
  - name: somename                                                                    
    rules:
      - alert: CertificateExpiry
        expr: certificate_expires_in_days < 20
        for: 1h
        labels:
          severity: critical
        annotations:
          summary: Certificate for {{ $labels.instance }} will expire in {{ $value }} days.   

and a test:

rule_files:
  - 'path/to/alert_rules.yml'
                                                                                         
evaluation_interval: 15m 

tests:
  - interval: 15m 
    input_series:
      - series: 'certificate_expires_in_days{instance="foo"}'
        values: '5 5 5 5 5 5 5 5'
    alert_rule_test:
      - eval_time: 2h
        alertname: CertificateExpiry
        exp_alerts:
          - exp_labels:
              severity: critical
              instance: foo 
            exp_annotations:
              summary: Certificate for foo will expire in 5 days.

The test fails, indicating the alert was not fired at all:

$ promtool test rules certificates.yml
Unit Testing:  certificates.yml
  FAILED:
    alertname:CertificateExpiry, time:2h, 
        exp:"[Labels:{alertname=\"CertificateExpiry\", instance=\"foo\", severity=\"critical\"} Annotations:{summary=\"Certificate for foo will expire in 5 days.\"}]", 
        got:"[]"

If, however, I change evaluation and series times like:

      - alert: CertificateExpiry                                                         
        expr: certificate_expires_in_days < 20                                           
        for: 8m                                                                          
        labels:                                                                          
          severity: critical                                                             
        annotations:                                                                     
          summary: Certificate for {{ $labels.instance }} will expire in {{ $value }} days. 

and

rule_files:
  - 'path/to/alert_rules.yml'

evaluation_interval: 1m                                                                  
                                                                                         
tests:                                                                                   
  - interval: 1m                                                                         
    input_series:                                                                        
      - series: 'certificate_expires_in_days{instance="foo"}'                            
        values: '5 5 5 5 5 5 5 5'                                                        
    alert_rule_test:                                                                     
      - eval_time: 8m                                                                    
        alertname: CertificateExpiry                                                     
        exp_alerts:                                                                      
          - exp_labels:                                                                  
              severity: critical                                                         
              instance: foo                                                              
            exp_annotations:                                                             
              summary: Certificate for foo will expire in 5 days.

the test succeeds. So, the problem is with times, the typos elsewhere can be excluded.

Apparently, it is some kind of a stupid attention mistake on my part, but I cannot see it. Could someone help, please?

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.