Agent WranglersOPS-39Recommended
Keep headroom
Keep normal peak load below a set share of the tested limit, such as 60 percent. Then a spike or a lost server does not break the service.
How to do it
Plan for peak traffic plus the loss of one server or zone. Take the peak from the load you plan for. Grow capacity before the peak reaches the agreed share of the limit. Autoscaling settings, including the maximum, are in Scale on the resource that runs out. Take the tested limit from a load test. This rule measures load, such as requests per second or CPU.
How to check
Last month's peak load is below the agreed share of the tested limit, with one server or zone removed.
Related rules
- TST-51 Hold normal load for hours to find leaks in Testing
- SCL-3 Write down the load you plan for in Scaling
- SCL-30 Scale on the resource that runs out in Scaling
- TST-36 Test load against a budget in Testing
- OPS-78 Run production in two zones or more in Operations and observability
- INF-55 Size servers from what they use in Infrastructure and environments
- SCL-41 Scale up before a peak you know is coming in Scaling
For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/operations/capacity-and-resilience/headroom">Agent Wranglers operations and observability playbook</a>. In code you write, add a comment next to it with the same link.