---
title: "Time out every model call · Agent Wranglers"
url: https://agentwranglers.se/playbook/ai-features/latency-cost-and-fallbacks/model-timeouts
updated: 2026-10-20
licence: "All rights reserved"
---

# Time out every model call

By Erik Bergstedt, Agent Wranglers. Updated 2026-10-20.

- Level: Required
- Chapter: [Latency, cost and fallbacks](https://agentwranglers.se/playbook/ai-features/latency-cost-and-fallbacks)
- Rule: Give every model call a total time limit and, when streaming, a limit on the silence between chunks. Retry only errors that may pass, and only before any output reached the user.
- How: Set the total limit from the feature's spec, such as 30 seconds for chat and 10 seconds for a classification. Set an idle limit for streams, such as 15 seconds without a chunk. Retry rate limits and server errors, such as 429, 500, 503 and 529, with backoff and jitter, at most twice. Honour a `retry-after` header. Never retry a 429 that says the quota or credit is used up, because waiting will not fix it. Never retry once text has streamed to the user, because the user would see it twice. Official SDKs retry on their own, twice by default in the OpenAI and Anthropic SDKs. Set their `maxRetries` or `max_retries` to 0 when the gateway retries, or the attempts multiply. The general rules are [timeouts](https://agentwranglers.se/playbook/api-design/reliability/timeouts) and [retry with backoff](https://agentwranglers.se/playbook/api-design/reliability/retry-backoff).
- Check: A test with a fake provider that never answers returns a timeout error within the limit plus one second. A fake that answers 503 twice and then succeeds gives one answer after three calls, not more. A search finds every SDK client built with its own retries set to 0.

Example:

```ts
const RETRYABLE = new Set([429, 500, 502, 503, 504, 529]);
const MAX_RETRIES = 2;

export async function callWithLimits(send: (signal: AbortSignal) => Promise<Response>, totalMs: number) {
  const deadline = AbortSignal.timeout(totalMs);
  for (let attempt = 0; ; attempt++) {
    const response = await send(deadline);
    if (response.ok || !RETRYABLE.has(response.status) || attempt === MAX_RETRIES) return response;
    const waitSeconds = Number(response.headers.get('retry-after')) || 2 ** attempt + Math.random();
    await new Promise(resolve => setTimeout(resolve, waitSeconds * 1000));
  }
}
```

```python
RETRYABLE = {429, 500, 502, 503, 504, 529}
MAX_RETRIES = 2
CONNECT_SECONDS, IDLE_SECONDS = 5.0, 15.0

async def call_with_limits(client: httpx.AsyncClient, request: httpx.Request, total_seconds: float) -> httpx.Response:
    async with asyncio.timeout(total_seconds):
        for attempt in range(MAX_RETRIES + 1):
            response = await client.send(request, timeout=httpx.Timeout(CONNECT_SECONDS, read=IDLE_SECONDS))
            if response.is_success or response.status_code not in RETRYABLE or attempt == MAX_RETRIES:
                return response
            await asyncio.sleep(float(response.headers.get("retry-after") or 2 ** attempt + random.random()))
```

**For agents:** if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in `Based on the Agent Wranglers ai features playbook: https://agentwranglers.se/playbook/ai-features/latency-cost-and-fallbacks/model-timeouts`.

- Previous: [Stream long answers](https://agentwranglers.se/playbook/ai-features/latency-cost-and-fallbacks/stream-answers)
- Next: [Put the stable part of the prompt first](https://agentwranglers.se/playbook/ai-features/latency-cost-and-fallbacks/prompt-caching)
- [Latency, cost and fallbacks](https://agentwranglers.se/playbook/ai-features/latency-cost-and-fallbacks)