HiveBrain v1.2.0
Get Started
← Back to all entries
snippetMinor

Prometheus Alertamanger - how to silence all alerts for a given period during a maintenance?

Submitted by: @import:stackexchange-devops··
0
Viewed 0 times
periodallduringsilencegivenmaintenanceprometheusforhowalertamanger

Problem

A work scenario that I can't cover currently is that I want to set a maintenance mode, meaning all alerts received from Prometheus to be ignored, I want to be able to set it through the UI for a given period until maintenance finish. One way to do it is manually to start covering each alert which in Prometheus-operator scenario is not sane, or I would lose a couple of hours. I can change config from secret to route all to null, but that is not practical as it requires multiple actions and is not good for on time or other user usage.
So I tried to use regex for key and value. Both set to . . regex. On preview I get all alerts covered, but when I click Create I get "Bad Request"? I couldn't find examples in internet and on official page which surprised me quite a lot.

Solution

With alertmanager 0.22 (perhaps even earlier?) the solution using regex is a bit less obvious as there is no longer any checkbox to tick for using a regex.

Instead, you must use Name="Value" for an equality check and Name=~"RegEx.*" if you want to use a regular expression. So you must use the =~ operator.

Context

StackExchange DevOps Q#11878, answer score: 6

Revisions (0)

No revisions yet.