---
title: "Define success before you build · Agent Wranglers"
url: https://agentwranglers.se/playbook/ai-features/deciding-to-use-a-model/define-ai-success
updated: 2026-10-20
licence: "All rights reserved"
---

# Define success before you build

By Erik Bergstedt, Agent Wranglers. Updated 2026-10-20.

- Level: Required
- Chapter: [Deciding to use a model](https://agentwranglers.se/playbook/ai-features/deciding-to-use-a-model)
- Rule: Before building an AI feature, write down its task, its target score, its worst possible mistake, its cost limit per use and its latency limit.
- How: Write a one-page spec. Name the task, the users and the inputs. Set a target score on an eval set, such as "90 of 100 answers correct". Name the failure that hurts most, such as "promises a refund we do not give". Set its limit, such as zero in the eval set. Set a cost ceiling per request and a latency limit, such as 3 seconds at the 95th percentile. A streamed answer also gets a limit on time to the first token. The [AI evals rule](https://agentwranglers.se/playbook/testing/non-functional-tests/llm-evals) turns the score into a test.
- Check: The spec file exists next to the feature's code and has a target score, a worst-failure limit, a cost per request and a latency limit. CI reads the target score from it.

Example:

```yaml
# ai/specs/return-assistant.yaml
task: Answer return and refund questions for Fernwood Bikes customers
target_score: 0.90
worst_failure: Promises a refund outside the 30-day policy
worst_failure_limit: 0
max_cost_per_request_usd: 0.01
p95_latency_ms: 3000
p95_first_token_ms: 1500
```

**For agents:** if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in `Based on the Agent Wranglers ai features playbook: https://agentwranglers.se/playbook/ai-features/deciding-to-use-a-model/define-ai-success`.

- Previous: [Try plain code before a model](https://agentwranglers.se/playbook/ai-features/deciding-to-use-a-model/model-or-code)
- Next: [Choose the smallest model that passes](https://agentwranglers.se/playbook/ai-features/deciding-to-use-a-model/smallest-passing-model)
- [Deciding to use a model](https://agentwranglers.se/playbook/ai-features/deciding-to-use-a-model)