---
title: "Load balancing and autoscaling · Agent Wranglers"
url: https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling
updated: 2026-10-21
licence: "CC BY 4.0"
licence_url: https://creativecommons.org/licenses/by/4.0/
---

# Load balancing and autoscaling

For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [Scaling](https://agentwranglers.se/playbook/scaling.md).

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers). Updated 2026-10-21.

> Send traffic only to ready servers and spread it by load. Scale on the resource that runs out, cap each customer's share and watch cost per request.

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling">Agent Wranglers scaling playbook</a>`. In code you write, add a comment next to it with the same link.

## Recommended

- SCL-29 [Spread load by requests in flight](https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling/least-requests-balancing): Send each request to the ready backend with the fewest requests or connections in flight, not strictly in turn. Take a backend out of rotation when connections to it fail.
- SCL-30 [Scale on the resource that runs out](https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling/scale-on-saturation): Drive autoscaling from the metric that limits each process type, such as CPU, requests in flight per copy or the age of the oldest job. Set a minimum, a maximum and a slow scale-down.
- SCL-31 [Give each customer a fair share](https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling/per-tenant-limits): Cap how much work each customer or tenant can have running or queued at once. One heavy customer then slows only their own work.
- SCL-32 [Turn off optional work under load](https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling/degrade-under-load): Mark the features the service can live without, such as recommendations or live counters. Keep a switch that turns them off under overload without a deploy.
- SCL-33 [Track cost per request](https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling/cost-per-request): Divide each service's monthly cost by the work it did, such as cost per 1,000 requests or per active customer. Alert when it rises more than 20% from the month before.

- Previous: [Growing the database](https://agentwranglers.se/playbook/scaling/growing-the-database)
- [All chapters](https://agentwranglers.se/playbook/scaling)