---
title: "Spread load by requests in flight · Agent Wranglers"
url: https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling/least-requests-balancing
updated: 2026-10-21
licence: "CC BY 4.0"
licence_url: https://creativecommons.org/licenses/by/4.0/
---

# Spread load by requests in flight

For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [Load balancing and autoscaling](https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling.md).

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers). Updated 2026-10-21.

- ID: SCL-29
- Level: Recommended
- Chapter: [Load balancing and autoscaling](https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling)
- Rule: Send each request to the ready backend with the fewest requests or connections in flight, not strictly in turn. Take a backend out of rotation when connections to it fail.
- Applies when: The project runs its own servers, virtual machines or containers.
- How: Requests differ in cost, so plain round robin leaves some servers far busier than others. Google measured up to 2 times more CPU on the busiest task than on the least busy. Most proxies offer a least-requests or least-connections mode, such as nginx `least_conn`. A backend that fails fast can attract more traffic in this mode, so count its errors and remove it after a set number. Readiness comes from [Separate alive from ready](https://agentwranglers.se/playbook/engineering/build-test-and-deploy/health-check-and-rollback). Draining on stop comes from [Shut down cleanly](https://agentwranglers.se/playbook/operations/running-the-service/graceful-shutdown).
- Check: Under the load test, the busiest backend's CPU is at most 1.3 times the least busy one's. Stop one backend's process. Within 30 seconds no request goes to it, and the error rate stays under 1%.

## Examples

**Using these examples:** they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

### nginx example

```nginx
upstream shop_app {
    least_conn;
    server 10.0.1.11:8080 max_fails=3 fail_timeout=10s;
    server 10.0.1.12:8080 max_fails=3 fail_timeout=10s;
    server 10.0.2.11:8080 max_fails=3 fail_timeout=10s;
}
```

## Sources

- [Google SRE book, Load Balancing in the Datacenter](https://sre.google/sre-book/load-balancing-datacenter/)
- [nginx upstream module](https://nginx.org/en/docs/http/ngx_http_upstream_module.html)

## Related rules

- [Give the server a health check](https://agentwranglers.se/playbook/engineering/build-test-and-deploy/health-check-and-rollback) in Engineering
- [Shut down cleanly](https://agentwranglers.se/playbook/operations/running-the-service/graceful-shutdown) in Operations

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling/least-requests-balancing">Agent Wranglers scaling playbook</a>`. In code you write, add a comment next to it with the same link.

- Next: [Scale on the resource that runs out](https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling/scale-on-saturation)
- [Load balancing and autoscaling](https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling)