AlertManager - Override configuration with helm

Astronomer
1

Hi
How do I deploy astronomer on k8s with overriding alertmanager configuration. Any example? I see Astronomer using alertmanager internally. I don’t see any reference entries in values.yaml.

2

I tried passing below lines for values.yaml. However the resultant alert manager configmap doesn’t show these rendered. Can someone show me what I missed?

#################################
## Alertmanager configuration
#################################
customRoutes:
    - receiver: astronomer-platform-critical
      match_re:
        tier: platform
        severity: warning
    - receiver: astronomer-airflow-critical
      match_re:
        tier: airflow
        severity: warning
customReceiver:
  - name: "astronomer-platform-critical"
    webhook_configs:
    - url: "https://prometheus-msteams:2000/astronomer-critical"
  - name: "astronomer-airflow-critical"
    webhook_configs:
    - url: "https://prometheus-msteams:2000/airflow-critival"

output alert manager configmap not considering customRoutes or customReceiver

################################
## Alertmanager ConfigMap
#################################
kind: ConfigMap
apiVersion: v1
metadata:
  name: astronomer-alertmanager
  labels:
    tier: monitoring
    component: alertmanager
    chart: alertmanager-0.12.0
    release: astronomer
    heritage: Helm
data:
  alertmanager.yaml: |-
    route:
      group_wait: 30s
      group_interval: 5m
      group_by: [alertname]
      repeat_interval: 3h
      receiver: default-receiver
      routes:

      
      - receiver:  blackhole-receiver 
        match_re:
          tier: platform
      - receiver:  default-receiver 
        group_by: [deployment, alertname]
        match_re:
          tier: airflow
      - receiver: blackhole-receiver
        match:
          silence: cre
    receivers:
    - name: blackhole-receiver
    # Deliberately left empty to not deliver anywhere.
    - name: default-receiver
      webhook_configs:
      - url: http://astronomer-houston:8871/v1/alerts
        send_resolved: true
3

Hey @VickyWinner,

I had a look into it and it looks like above mentioned altermanager configuration in your values.yaml are bit off. Could you please try below config:

alertmanager:
  customRoutes:
    - receiver: astronomer-platform-critical
      match_re:
        tier: platform
  receivers:
  # Configs for platform alerts
  platform: 
    email_configs:
      - smarthost: smtp.sendgrid.net:587
        from: <your-astronomer-alert-email@company.com>
        to: <your-email@company.com>
        auth_username: apikey
        auth_password: SG.myapikey1234567891234abcdef_bKY
        send_resolved: true
  platformCritical:
    slack_configs:
      - api_url: https://hooks.slack.com/services/abc12345/abcXYZ/xyz67890
        channel: '#<your-slack-channel-name>'
        text: |-
          {{ range .Alerts }}{{ .Annotations.description }}
          {{ end }}
        title: '{{ .CommonAnnotations.summary }}'

Also, please find below Astronomer software alerts documentation for your reference:

1 Like
4

Thanks for the response. I fixed that by adding alertmanager: and it worked. Just trying to add the webhook_configs with no success yet. Is it possible to send alerts to msteams natively?

5

looks like its not natively available