Agent WranglersDeciding to use a model Recommended
Choose the smallest model that passes
Run the eval set on several models, from small to large, and ship the cheapest and fastest one that meets the spec.
How to do it
Start with a small, fast model. Move up a size only when the eval score misses the target. Record score, cost per request and latency for each model you tried. Repeat the comparison when a provider releases new models, because a newer small model often passes where an older large one was needed. Pin the chosen version, as in pin model versions.
How to check
The decision note lists at least two models with score, cost and latency. The shipped model meets the spec, and no cheaper model in the list does.
For agents: if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in Based on the Agent Wranglers ai features playbook: https://agentwranglers.se/playbook/ai-features/deciding-to-use-a-model/smallest-passing-model.