Skip to main content

Alertmanager

Checks Prometheus AlertManager for any firing alerts.

The following example transforms the list of alerts so that each alert becomes a single check result. Without the transform the health check fails if any alerts are firing.

alert-manager-check.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
name: alertmanager
spec:
schedule: "@every 5m"
alertmanager:
- url: https://alertmanager.demo.aws.flanksource.com
name: alertmanager-check
alerts:
- .*
ignore:
- KubeScheduler.*
exclude_filters:
namespace: elastic-system
transform:
expr: |
results.alerts.map(r, {
'name': r.name + r.fingerprint,
'namespace': 'namespace' in r.labels ? r.labels.namespace : '',
'labels': r.labels,
'icon': 'alert',
'message': r.message,
'description': r.message,
}).toJSON()
relationships:
components:
- name:
label: pod
namespace:
label: namespace
type:
value: KubernetesPod
configs:
- name:
label: pod
namespace:
label: namespace
type:
value: Kubernetes::Pod
FieldDescriptionScheme
name*

Name of the check, must be unique within the canary

string

url*

The URL of the Prometheus Alertmanager instance

string

alerts

A list of alert prefix names to include

[]string

filters

A map of label to value prefixes to find alerts on

map[string]string

ignore

A list of alert prefix names to exclude

[]string

description

Description for the check

string

display

Expression to change the formatting of the display

Expression

icon

Icon for overwriting default icon on the dashboard

Icon

labels

Labels for check

map[string]string

metrics

Metrics to export from

[]Metrics

test

Evaluate whether a check is healthy

Expression

transform

Transform data from a check into multiple individual checks

Expression

Relationships

FieldDescriptionScheme
components

Select all the components to link to this check

Lookup

config

Select all the configs to link to this check

Lookup

Lookup

FieldDescriptionScheme
name

Initial delay before the checks are run

Lookup Selector

namespace

Timeout for the check

Lookup Selector

type

Timeout for the check

Lookup Selector

Lookup selector

Lookup specifies the type of lookup to perform.

FieldDescriptionScheme
expr

Expr is a cel-expression

string

value

Value is the static value to use

string

value

Label specifies the key to lookup on the label

string

Inserting checks into different namespaces

You can specify different namespaces for checks using the namespace field. This is helpful when checks are dynamically generated via transformation and need to be assigned their respective namespace.

alert-manager-check.yaml
apiVersion: canaries.flanksource.com/v1
kind: Canary
metadata:
name: alertmanager
spec:
schedule: "@every 5m"
alertmanager:
- url: https://alertmanager.demo.aws.flanksource.com
name: alertmanager-check
alerts:
- .*
ignore:
- KubeScheduler.*
exclude_filters:
namespace: elastic-system
transform:
expr: |
results.alerts.map(r, {
'name': r.name + r.fingerprint,
'namespace': 'namespace' in r.labels ? r.labels.namespace : '',
'labels': r.labels,
'icon': 'alert',
'message': r.message,
'description': r.message,
}).toJSON()
relationships:
components:
- name:
label: pod
namespace:
label: namespace
type:
value: KubernetesPod
configs:
- name:
label: pod
namespace:
label: namespace
type:
value: Kubernetes::Pod

The above alertmanager canary can reside on any namespace and the check it generates are assigned the correct namespace derived from the alerts.