Agent Wranglers

Recommended

Keep generation settings in the prompt file

Store the output limit and the generation settings, such as temperature or reasoning effort, in the prompt file's front matter. Raise the prompt version when any of them changes.

Applies when

The project it uses AI models.

How to do it

The same prompt gives different answers, costs and speed at another temperature or effort level. Keep the settings next to the text, so the eval run and production use the same ones. The gateway reads them from the file and ignores settings passed by feature code. Names and values differ by provider and model. The OpenAI Responses API takes reasoning.effort, with values from none to max, and not every model accepts every value. The Anthropic Messages API takes output_config.effort and a thinking setting, and thinking tokens count toward max_tokens. Run the eval set when a setting changes, as review and test every prompt change says. Keep the model name where pin the AI model version puts it.

Examples

Using these examples: they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

Markdown example

---
id: return-assistant
version: 8
schema: ReturnAnswer
settings:
  max_output_tokens: 800
  reasoning_effort: low
---
You answer questions about returns for Fernwood Bikes.

TypeScript example

const REQUIRED_SETTINGS = ['max_output_tokens', 'reasoning_effort'] as const;
for (const prompt of loadPromptFiles('ai/prompts')) {
  for (const name of REQUIRED_SETTINGS) expect(prompt.settings).toHaveProperty(name);
}

Python example

REQUIRED_SETTINGS = {"max_output_tokens", "reasoning_effort"}
for prompt in load_prompt_files("ai/prompts"):
    missing = REQUIRED_SETTINGS - prompt.settings.keys()
    assert not missing, f"{prompt.id} lacks {missing}"

How to check

A test loads every prompt file and finds an output limit and every generation setting the gateway sends. A search of feature code finds no model call that sets temperature, effort or an output limit.

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/ai-features/prompts-as-code/settings-with-prompt">Agent Wranglers AI features playbook</a>. In code you write, add a comment next to it with the same link.