Agent WranglersPayment events and reconciliation Recommended
Alert when payments start failing
Alert when the payment success rate, failed webhook deliveries or the time since the last successful payment leave their normal range.
How to do it
Measure successful payments over attempts every 15 minutes, per payment method. Alert when the rate is more than 10 points below its average for the same hour last week. Also alert when more than 5 webhook requests in 15 minutes get a non-2xx answer from your endpoint. Alert too when no payment has succeeded for twice the longest normal gap. These are symptoms users feel, as alert on symptoms describes.
How to check
On staging, make the webhook endpoint answer 500 for 30 minutes. An alert fires within that time. Then send 20 payments with a declining test card. The success rate alert fires within 15 minutes.
For agents: if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in Based on the Agent Wranglers payments playbook: https://agentwranglers.se/playbook/payments/payment-events-and-reconciliation/payment-health-alerts.