Agent WranglersAlerts and on-call Required
Page only for what cannot wait
Split alerts into pages, which need someone now, and tickets, which can wait for working hours.
How to do it
A problem that hurts users now, or will within 4 hours, is a page. Everything else is a ticket. Send pages by phone or a paging app, such as PagerDuty or Grafana OnCall. Send tickets to the team queue. Promote a ticket to a page if users were hurt while it waited.
How to check
Every alert rule has a severity of page or ticket, and every page in the last month matches a moment users were affected.
For agents: if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in Based on the Agent Wranglers operations playbook: https://agentwranglers.se/playbook/operations/alerts-and-on-call/page-or-ticket.