---
title: "Turn work away before overload · Agent Wranglers"
url: https://agentwranglers.se/playbook/operations/capacity-and-cost/load-shedding
updated: 2026-10-20
licence: "All rights reserved"
---

# Turn work away before overload

By Erik Bergstedt, Agent Wranglers. Updated 2026-10-20.

- Level: Recommended
- Chapter: [Capacity and cost](https://agentwranglers.se/playbook/operations/capacity-and-cost)
- Rule: Cap how many requests each instance handles at once. Answer the rest at once with 503 and a `Retry-After` header instead of queueing them.
- How: Under overload, a long queue makes every request slow until all of them time out, and nobody is served. Set the cap from a load test, near the number of requests in flight where response time starts to climb. Register the health check before the cap, so it still answers. Per-client limits are a separate rule, [rate limits](https://agentwranglers.se/playbook/website/security/rate-limits). Clients wait and retry as in [retry with backoff](https://agentwranglers.se/playbook/api-design/reliability/retry-backoff).
- Check: Load staging with twice its tested limit. Extra requests get 503 within 100 ms, accepted requests stay within their speed objective, and the service recovers without a restart.

Example:

```js
const MAX_IN_FLIGHT = 200;
let inFlight = 0;

app.use((req, res, next) => {
  if (inFlight >= MAX_IN_FLIGHT) return res.set('Retry-After', '5').status(503).end();
  inFlight += 1;
  res.on('close', () => { inFlight -= 1; });
  next();
});
```

```python
from starlette.responses import Response

MAX_IN_FLIGHT = 200
in_flight = 0

@app.middleware("http")
async def shed_load(request, call_next):
    global in_flight
    if in_flight >= MAX_IN_FLIGHT:
        return Response(status_code=503, headers={"Retry-After": "5"})
    in_flight += 1
    try:
        return await call_next(request)
    finally:
        in_flight -= 1
```

**For agents:** if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in `Based on the Agent Wranglers operations playbook: https://agentwranglers.se/playbook/operations/capacity-and-cost/load-shedding`.

- Previous: [Keep headroom](https://agentwranglers.se/playbook/operations/capacity-and-cost/headroom)
- Next: [Size the runtime's memory to the box](https://agentwranglers.se/playbook/operations/capacity-and-cost/runtime-memory-limits)
- [Capacity and cost](https://agentwranglers.se/playbook/operations/capacity-and-cost)