Route each kind of request to the model it needs

How to do it

Choose the smallest model that passes picks one model per feature. Routing applies the same test to each kind of request inside a feature. A short question can go to a small model, and a long contract review to a large one. Start with plain rules, such as the task type, the input length or the customer's plan. A model that sorts requests is one more model to evaluate and pay for. Send a request on to the larger model when the small model's answer fails its schema or output checks, and count how often that happens. Keep the route table in the module from Call models through one module. Record the cost per route as Price every call and cap the spend does. A provider outage is handled by Keep working when the provider fails, not by routing.

Examples

TypeScript example

const ROUTES = {
  short_question: { model: 'example-small-2026-05-14', maxInputTokens: 2_000 },
  contract_review: { model: 'example-large-2026-05-14', maxInputTokens: 150_000 },
} as const;

export function chooseRoute(task: 'short_question' | 'contract_review', inputTokens: number) {
  const route = inputTokens > ROUTES.short_question.maxInputTokens ? 'contract_review' : task;
  return { route, model: ROUTES[route].model };
}

Python example

ROUTES = {
    "short_question": {"model": "example-small-2026-05-14", "max_input_tokens": 2_000},
    "contract_review": {"model": "example-large-2026-05-14", "max_input_tokens": 150_000},
}

def choose_route(task: str, input_tokens: int) -> dict[str, str]:
    route = "contract_review" if input_tokens > ROUTES["short_question"]["max_input_tokens"] else task
    return {"route": route, "model": ROUTES[route]["model"]}

How to check

The eval report gives a score per route, and each route's model meets the spec on that route's cases. A unit test sends one request of each kind and asserts the chosen model. Every model call log line has a route field.

Sources

• Claude Docs, choosing the right model

Related rules

• AIF-3 Choose the smallest model that passes in AI features
• AIF-4 Call models through one module in AI features
• AIF-42 Price every call and cap the spend in AI features
• AIF-44 Keep working when the provider fails in AI features