Keep the tool list within a token budget

How to do it

Clients put every tool definition into the model's context on every turn. A 40-tool server with long descriptions costs each user thousands of tokens before any work starts. Pick a budget, such as 8,000 tokens, and write it in the repository. Merge tools that differ only by a filter into one tool with an enum argument. Move long reference text into a resource the tool links to. Return only the tools the caller's scopes allow, which the specification permits. Name and shape each tool as Design tools like a small public API says. Measure the effect of each change with Keep a set of tasks with the tools they need.

Examples

JavaScript example

import { encode } from 'gpt-tokenizer';
const TOOL_LIST_TOKEN_BUDGET = 8000;

test('tool list fits its budget', async () => {
  const { tools } = await client.listTools();
  expect(encode(JSON.stringify(tools)).length).toBeLessThan(TOOL_LIST_TOKEN_BUDGET);
  tools.forEach(tool => expect(tool.description).toMatch(/Use when|Do not use/));
});

Python example

import json
import tiktoken

TOOL_LIST_TOKEN_BUDGET = 8000

def test_tool_list_fits_its_budget(client):
    tools = client.list_tools()["tools"]
    tokens = len(tiktoken.get_encoding("o200k_base").encode(json.dumps(tools)))
    assert tokens < TOOL_LIST_TOKEN_BUDGET
    assert all("Use when" in tool["description"] or "Do not use" in tool["description"] for tool in tools)

How to check

A test calls tools/list with a token that holds every scope and counts the tokens of the JSON answer with a common tokenizer. The count is under the budget written in the repository. Every description has a "Use when" or "Do not use" sentence.

Sources

• MCP specification on tools

Related rules

• AIF-15 Design tools like a small public API in AI features
• AGT-28 Keep a set of tasks with the tools they need in Agents as users
• AGT-31 Rerun the tool evals on every tool change in Agents as users