---
title: "Keep working when the provider fails · Agent Wranglers"
url: https://agentwranglers.se/playbook/ai-features/latency-cost-and-fallbacks/provider-fallback
updated: 2026-10-20
licence: "All rights reserved"
---

# Keep working when the provider fails

By Erik Bergstedt, Agent Wranglers. Updated 2026-10-20.

- Level: Required
- Chapter: [Latency, cost and fallbacks](https://agentwranglers.se/playbook/ai-features/latency-cost-and-fallbacks)
- Rule: Decide for each AI feature what happens when the model provider is down, slow or out of quota, and test that path. Use a second model that passed the same evals, a version without AI, or a clear message, never a hang or a broken page.
- How: Choose one fallback per feature. Options are a second model from another provider or region, a smaller model, a cached answer or the feature without AI, such as plain search. Run the eval set on the fallback model too, because it will answer differently. Switch after a set number of failures in a row, such as 5. Try the main model again after a pause, such as 60 seconds. Show users a note when the fallback is in use. Retries before the switch follow [model timeouts](https://agentwranglers.se/playbook/ai-features/latency-cost-and-fallbacks/model-timeouts). The operations side, alerts and the rest of the product, is in [AI provider failure](https://agentwranglers.se/playbook/operations/disaster-recovery/model-provider-failure).
- Check: In staging, block the main provider's address. Each AI feature still answers or shows its non-AI version within its time limit, and the logs show the fallback in use.

Example:

```ts
export async function generateWithFallback<T>(request: GenerateRequest<T>): Promise<T & { servedBy: string }> {
  if (!breaker.isOpen('primary')) {
    try { return { ...(await primary.generate(request)), servedBy: 'primary' }; }
    catch (error) { breaker.recordFailure('primary'); log.warn({ event: 'ai.primary_failed', error: String(error) }); }
  }
  return { ...(await fallback.generate(request)), servedBy: 'fallback' };
}
```

```python
def generate_with_fallback(request: GenerateRequest) -> Result:
    if not breaker.is_open("primary"):
        try:
            return primary.generate(request).with_served_by("primary")
        except ProviderError as error:
            breaker.record_failure("primary")
            log.warning("ai.primary_failed", error=str(error))
    return fallback.generate(request).with_served_by("fallback")
```

**For agents:** if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in `Based on the Agent Wranglers ai features playbook: https://agentwranglers.se/playbook/ai-features/latency-cost-and-fallbacks/provider-fallback`.

- Previous: [Keep every request inside a token budget](https://agentwranglers.se/playbook/ai-features/latency-cost-and-fallbacks/context-budget)
- Next: [Send work nobody waits for as a batch](https://agentwranglers.se/playbook/ai-features/latency-cost-and-fallbacks/batch-offline-work)
- [Latency, cost and fallbacks](https://agentwranglers.se/playbook/ai-features/latency-cost-and-fallbacks)