--- title: "Ready for AI agents · Agent Wranglers" url: https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents parts: 37 tokens_total: 28392 updated: 2026-10-27 build: a2a6bdfed830 tokens: 3131 licence: "CC BY 4.0" licence_url: https://creativecommons.org/licenses/by/4.0/ --- # Ready for AI agents For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [Agents as users](https://agentwranglers.se/playbook/agents-as-users.md). By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers.md). Updated 2026-10-27. > Markdown copies, llms.txt, AI crawler rules, structured data, link headers, MCP, A2A, skills and WebMCP make a site easy for AI agents to use. **For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `Agent Wranglers agents as users playbook`. In code you write, add a comment next to it with the same link. ## Recommended - AGT-42 [Make the site easy for AI agents](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/agent-readiness-overview.md) · 588 tokens · updated 2026-10-27 Make the site easy for AI agents to read and use. Put content in the HTML, and add structured data, Markdown copies, clear crawler rules and discovery files. - AGT-49 [Offer every page as Markdown](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/markdown-source-endpoints.md) · 983 tokens · updated 2026-10-27 Serve every page as Markdown too. Use its address plus `.md`, and answer requests with `Accept: text/markdown`. Point back to the HTML as the canonical copy. - AGT-50 [Link Markdown copies to Markdown copies](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/markdown-links-to-markdown.md) · 738 tokens · updated 2026-10-27 Every internal link in a Markdown copy points to the Markdown copy of its target. - AGT-79 [Mark user content that agents read](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/mark-user-content.md) · 680 tokens · updated 2026-10-27 Keep text written by your users apart from your own content, so other people's agents can tell it is not yours. - AGT-52 [Decide what each AI crawler may do](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/robots-for-ai-crawlers.md) · 1,147 tokens · updated 2026-10-27 Decide what each AI crawler may do. Write it in robots.txt as its own user-agent group. - AGT-53 [Let invited agents reach what you invite them to](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/robots-allow-invited-endpoints.md) · 819 tokens · updated 2026-10-27 robots.txt never blocks an endpoint that llms.txt, a skill or an agent card tells agents to use. - AGT-59 [Offer content machines can read](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/machine-readable-formats.md) · 897 tokens · updated 2026-10-27 Offer the site's content as a sitemap, feeds, Markdown and JSON-LD. - AGT-72 [Answer bursts with 429 instead of a stall](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/burst-answers.md) · 740 tokens · updated 2026-10-27 When a client asks too often, answer 429 with `Retry-After` at once. Serve large agent files as files built ahead of time, never built on request. - AGT-73 [Resolve retired IDs everywhere](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/resolve-retired-ids.md) · 615 tokens · updated 2026-10-27 Every place that accepts an ID follows the map of moved IDs. A lookup of several IDs returns what it found, what moved and what it did not find. - AGT-74 [Use one name for each thing in every agent file](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/one-name-everywhere.md) · 775 tokens · updated 2026-10-27 Give each section, product or document one name. Use it in the page title, the `h1`, the front matter, llms.txt, the JSON files and the agent cards. - AGT-75 [Keep feeds to recent changes](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/recent-changes-feed.md) · 531 tokens · updated 2026-10-27 A site feed carries recent updates only, with a summary and a link per entry, not the full text of every page. ## Optional - AGT-43 [Give AI tools a site guide](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/llms-txt.md) · 1,203 tokens · updated 2026-10-27 Publish `/llms.txt`, a short Markdown guide to your site for AI tools. Give the site's name, a one-line summary, and links to the Markdown version of each important page. - AGT-44 [Offer one file only while it fits an agent's context](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/llms-full-txt.md) · 925 tokens · updated 2026-10-27 Offer all content in one Markdown file, such as `/llms-full.txt`, only while it stays under 100,000 tokens. Above that, split it by section behind an index. - AGT-46 [State the size of every agent file in tokens](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/token-counts.md) · 783 tokens · updated 2026-10-27 Every file for agents states its size in tokens, so an agent knows the cost before it fetches the file. - AGT-47 [Layer agent content with a budget per layer](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/layer-budgets.md) · 717 tokens · updated 2026-10-27 Arrange agent content in layers, each with a token budget, and link each layer to the next as Markdown. - AGT-48 [Split large agent content into named parts behind an index](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/named-parts-index.md) · 915 tokens · updated 2026-10-27 Split content too large for one read into parts by meaning. An index lists every part with a summary, a size in tokens, an address and a date. - AGT-54 [Say how your content may be used](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/content-signals.md) · 778 tokens · updated 2026-10-27 Use `Content-Signal` lines in robots.txt to say whether your content may be used for search, AI answers or AI training. - AGT-55 [State your text-mining choice](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/tdmrep.md) · 662 tokens · updated 2026-10-27 State whether you reserve text-and-data-mining rights. Use a `tdm-reservation` header and `/.well-known/tdmrep.json`, matching robots.txt. - AGT-56 [Sign your bot's requests](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/web-bot-auth.md) · 738 tokens · updated 2026-10-27 If you run a crawler or agent, sign its requests with Web Bot Auth. As a site, don't block unsigned bots by default. - AGT-57 [Explain how agents sign up](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/auth-md.md) · 713 tokens · updated 2026-10-27 If agents must register before using your service, explain how in an `auth.md` file at the root of your service. - AGT-58 [Publish each page's data as a file](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/structured-data-for-agents.md) · 652 tokens · updated 2026-10-27 Publish each page's JSON-LD (structured facts) in the page and as its own file. Then agents can read the facts without parsing HTML. - AGT-60 [Advertise agent files in headers](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/link-headers.md) · 705 tokens · updated 2026-10-27 List your discovery files in HTTP `Link` headers on every response. These include llms.txt, the sitemap, feeds, the API catalog and agent cards. - AGT-61 [Publish a server card for your MCP server](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/mcp-and-tool-discovery.md) · 1,054 tokens · updated 2026-10-27 If you run an MCP server, a way for AI assistants to use your tools, publish a server card that lists it. - AGT-62 [Publish an agent card](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/a2a-agent-cards.md) · 737 tokens · updated 2026-10-27 If you run an A2A agent, one that talks to other agents, publish its card at `/.well-known/agent-card.json`. Keep a working endpoint behind it. - AGT-63 [List skills agents can learn](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/agent-skills-discovery.md) · 805 tokens · updated 2026-10-27 Publish agent skills as `SKILL.md` files, listed in `/.well-known/agent-skills/index.json`. Give each one a digest, a hash of its exact bytes. - AGT-64 [List agent endpoints in DNS](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/dns-aid.md) · 676 tokens · updated 2026-10-27 List your agent endpoints in DNS with SVCB records, as the DNS-AID draft describes. Sign the zone with DNSSEC so answers can be trusted. - AGT-65 [Keep one catalogue of agent tools](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/agentic-resource-discovery.md) · 778 tokens · updated 2026-10-27 List your MCP servers, agents and skills in `/.well-known/ai-catalog.json`. Link it from every page. - AGT-66 [Answer questions from your own pages](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/nlweb.md) · 969 tokens · updated 2026-10-27 Offer an endpoint that answers plain-language questions from your own content, with sources. - AGT-67 [Publish a topic index for agents](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/topic-index.md) · 680 tokens · updated 2026-10-27 Offer one small file that maps common topics to the pages that answer them. - AGT-68 [Give in-browser agents your tools](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/webmcp.md) · 848 tokens · updated 2026-10-27 Register the site's tools with the browser through WebMCP. Mark the ones with effects, and report expected failures in the tool's answer. - AGT-69 [Package your knowledge for agents](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/okf-bundle.md) · 543 tokens · updated 2026-10-27 Publish the site's knowledge as an Open Knowledge Format bundle. It has an index, one typed Markdown file per concept, and a change log. - AGT-70 [Index every page's data file](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/schemamap.md) · 539 tokens · updated 2026-10-27 List every JSON-LD file with its types in `/schemamap.xml`. Link it from the page head. - AGT-71 [Show one credit line to every reader](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/ask-for-credit.md) · 750 tokens · updated 2026-10-27 If others may reuse your content, show one visible credit line on every page, with the same words for people and agents. Repeat it in the Markdown copy. - AGT-76 [Put the answer first in every Markdown copy](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/answer-first-markdown.md) · 672 tokens · updated 2026-10-27 A Markdown copy opens with front matter and then the main point. It repeats no text that appears on every page. - AGT-77 [Measure how much an agent reads to find an answer](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/agent-path-tests.md) · 695 tokens · updated 2026-10-27 Keep a fixed set of questions with their expected pages. Measure the tokens and fetches a fresh agent needs to reach each one. ## Avoid - AGT-45 [Serving one file too large for any agent to read](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/oversized-agent-file.md) · 546 tokens · updated 2026-10-27 Never publish a file for agents to read that is larger than an agent's context window. A full-site dump of millions of tokens is one example. - AGT-51 [Showing agents a different page than people](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/same-content-for-agents.md) · 796 tokens · updated 2026-10-27 Never serve AI crawlers or agents different content from what people see, chosen by user agent, IP address or bot signature. ## Read this with - [Website playbook](https://agentwranglers.se/playbook/website.md). AI tools read sites too. Give them Markdown copies, a site guide and clear rules for each AI crawler. - [SEO playbook](https://agentwranglers.se/playbook/seo.md). AI tools read sites too. Give them Markdown copies, a site guide and clear rules for each AI crawler. - Previous: [Quotas and pricing](https://agentwranglers.se/playbook/agents-as-users/quotas-and-pricing.md) - [All chapters](https://agentwranglers.se/playbook/agents-as-users.md)