---
title: "Check every model answer against a schema · Agent Wranglers"
url: https://agentwranglers.se/playbook/adversarial-testing/ai-agents-and-models/validate-model-output
updated: 2026-10-20
licence: "All rights reserved"
---

# Check every model answer against a schema

By Erik Bergstedt, Agent Wranglers. Updated 2026-10-20.

- Level: Required
- Chapter: [AI agents and models](https://agentwranglers.se/playbook/adversarial-testing/ai-agents-and-models)
- Rule: Parse every model answer your code acts on against a schema before you use it. Reject, retry or fall back when it does not match.
- How: Ask for structured output where the provider supports it, and still validate it, for example with a JSON Schema validator, zod or pydantic. Check value ranges and allowed options, not only the shape. Log the answers that fail, without personal data. See also [retry a bad answer and then fall back](https://agentwranglers.se/playbook/ai-features/structured-output-and-tools/retry-invalid-output).
- Check: Make a test model return text instead of JSON, then JSON with an unknown option. Both are rejected, and nothing downstream runs.

Example:

```js
const Refund = z.object({ orderId: z.string(), amount: z.number().positive().max(500), reason: z.enum(['damaged', 'late', 'wrong-item']) });
const parseJson = text => { try { return JSON.parse(text); } catch { return undefined; } };
const parsed = Refund.safeParse(parseJson(answer));
if (!parsed.success) return askAPersonInstead(answer);
```

```python
# Python with pydantic
class Refund(BaseModel):
    order_id: str
    amount: float = Field(gt=0, le=500)
    reason: Literal["damaged", "late", "wrong-item"]

try:
    refund = Refund.model_validate_json(answer)
except ValidationError:
    return ask_a_person_instead(answer)
```

**For agents:** if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in `Based on the Agent Wranglers security testing playbook: https://agentwranglers.se/playbook/adversarial-testing/ai-agents-and-models/validate-model-output`.

- Previous: [Send models only the private data they need](https://agentwranglers.se/playbook/adversarial-testing/ai-agents-and-models/private-data-to-models)
- Next: [Keep a switch that turns the agent off](https://agentwranglers.se/playbook/adversarial-testing/ai-agents-and-models/agent-kill-switch)
- [AI agents and models](https://agentwranglers.se/playbook/adversarial-testing/ai-agents-and-models)