Retry with backoff and jitter

How to do it

Retry on network errors, 408, 429, 502, 503 and 504. Never retry other 4xx answers. Start at about 200 milliseconds, double the cap each time and wait a random time up to that cap. Stop after 5 attempts or 30 seconds. When Retry-After is present, wait at least that long. Calls a person waits for retry fewer times, as in screen request time limits and model timeouts. Background jobs move to a dead letter queue after their limit. Follow this policy in your own SDKs and document it for clients.

Example

const RETRYABLE = new Set([408, 429, 502, 503, 504])

async function fetchWithRetry(url, options, maxAttempts = 5, baseMs = 200) {
  for (let attempt = 1; ; attempt++) {
    const response = await fetch(url, options).catch(() => null)
    if (response && !RETRYABLE.has(response.status)) return response
    if (attempt === maxAttempts) return response ?? Promise.reject(new Error('network error'))
    const retryAfterMs = Number(response?.headers.get('retry-after')) * 1000 || 0
    const backoffMs = Math.random() * baseMs * 2 ** attempt
    await new Promise(resolve => setTimeout(resolve, Math.max(retryAfterMs, backoffMs)))
  }
}

import random, time
import httpx

RETRYABLE = {408, 429, 502, 503, 504}

def request_with_retry(client: httpx.Client, method, url, max_attempts=5, base_s=0.2, **kwargs):
    for attempt in range(1, max_attempts + 1):
        try:
            response = client.request(method, url, **kwargs)
            if response.status_code not in RETRYABLE or attempt == max_attempts:
                return response
            header = response.headers.get("retry-after", "")
            retry_after = float(header) if header.isdigit() else 0
        except httpx.TransportError:
            if attempt == max_attempts:
                raise
            retry_after = 0
        time.sleep(max(retry_after, random.uniform(0, base_s * 2 ** attempt)))

How to check

A test server answers 503 three times. The client retries with growing gaps, honours Retry-After and stops at the cap. A 400 is never retried.