---
title: "Turn work away before overload · Agent Wranglers"
url: https://agentwranglers.se/playbook/operations/capacity-and-resilience/load-shedding
id: OPS-40
level: Recommended
updated: 2026-10-27
build: a2a6bdfed830
tokens: 1153
licence: "CC BY 4.0"
licence_url: https://creativecommons.org/licenses/by/4.0/
---

# Turn work away before overload

For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [Capacity and resilience](https://agentwranglers.se/playbook/operations/capacity-and-resilience.md).

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers.md). Updated 2026-10-27.

- ID: OPS-40
- Level: Recommended
- Chapter: [Capacity and resilience](https://agentwranglers.se/playbook/operations/capacity-and-resilience.md)
- Rule: Cap how many requests each instance handles at once. Answer the rest at once with 503 and a `Retry-After` header instead of queueing them.
- Check: Load staging with twice its tested limit. Extra requests get 503 within 100 ms, accepted requests stay within their speed objective, and the service recovers without a restart.
- How: Under overload, a long queue makes every request slow until all of them time out, and nobody is served. Set the cap from a load test, near the number of requests in flight where response time starts to climb. Register the health check before the cap, so it still answers. Before turning requests away, switch off optional work, as [Turn off optional work under load](https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling/degrade-under-load.md) says. Per-client limits are a separate rule, [rate limits](https://agentwranglers.se/playbook/application-security/limits-and-business-logic/rate-limits.md). Clients wait and retry as in [retry with backoff](https://agentwranglers.se/playbook/api-design/reliability/retry-backoff.md).

## Examples

**Using these examples:** they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

### JavaScript example

```js
const MAX_IN_FLIGHT = 200;
let inFlight = 0;

app.use((req, res, next) => {
  if (inFlight >= MAX_IN_FLIGHT) return res.set('Retry-After', '5').status(503).end();
  inFlight += 1;
  res.on('close', () => { inFlight -= 1; });
  next();
});
```

### Python example

```python
from starlette.responses import Response

MAX_IN_FLIGHT = 200
in_flight = 0

@app.middleware("http")
async def shed_load(request, call_next):
    global in_flight
    if in_flight >= MAX_IN_FLIGHT:
        return Response(status_code=503, headers={"Retry-After": "5"})
    in_flight += 1
    try:
        return await call_next(request)
    finally:
        in_flight -= 1
```

## Sources

- [RFC 9110 section 15.6.4 503 Service Unavailable](https://www.rfc-editor.org/rfc/rfc9110#section-15.6.4)
- [RFC 9110 section 10.2.3 Retry-After](https://www.rfc-editor.org/rfc/rfc9110#section-10.2.3)

## Related rules

- SCL-32 [Turn off optional work under load](https://agentwranglers.se/playbook/scaling/load-balancing-and-autoscaling/degrade-under-load.md) in Scaling
- APS-49 [Limit how often anyone can call your APIs](https://agentwranglers.se/playbook/application-security/limits-and-business-logic/rate-limits.md) in Application security
- API-34 [Retry with backoff and jitter](https://agentwranglers.se/playbook/api-design/reliability/retry-backoff.md) in API design
- OPS-66 [Cap retries across the whole service](https://agentwranglers.se/playbook/operations/capacity-and-resilience/retry-budget.md) in Operations and observability
- OPS-74 [Stop calling a failing service with a circuit breaker](https://agentwranglers.se/playbook/operations/capacity-and-resilience/circuit-breakers.md) in Operations and observability
- SCL-15 [Keep the app working when the cache is gone](https://agentwranglers.se/playbook/scaling/caching/survive-cache-loss.md) in Scaling
- EVT-18 [Bound the backlog and push back on producers](https://agentwranglers.se/playbook/events/background-jobs-and-queues/bounded-backlog.md) in Integration and events

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/operations/capacity-and-resilience/load-shedding">Agent Wranglers operations and observability playbook</a>`. In code you write, add a comment next to it with the same link.

- Previous: [Keep headroom](https://agentwranglers.se/playbook/operations/capacity-and-resilience/headroom.md)
- Next: [Match timeouts from proxy to app and stop abandoned work](https://agentwranglers.se/playbook/operations/capacity-and-resilience/timeout-chain.md)
- [Capacity and resilience](https://agentwranglers.se/playbook/operations/capacity-and-resilience.md)