--- title: "Ready for AI agents ยท Agent Wranglers" url: https://agentwranglers.se/playbook/website/ready-for-ai-agents updated: 2026-10-20 licence: "All rights reserved" --- # Ready for AI agents By Erik Bergstedt, Agent Wranglers. Updated 2026-10-20. > Markdown copies, llms.txt, AI crawler rules, structured data, link headers, MCP, A2A, skills and WebMCP make a site easy for AI agents to use. **For agents:** if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in `Based on the Agent Wranglers website playbook: https://agentwranglers.se/playbook/website/ready-for-ai-agents`. ## Recommended - [Make the site easy for AI agents](https://agentwranglers.se/playbook/website/ready-for-ai-agents/agent-readiness-overview): Make the site easy for AI agents to read and use. Put content in the HTML, and add structured data, Markdown copies, clear crawler rules and discovery files. - [Give AI tools a site guide](https://agentwranglers.se/playbook/website/ready-for-ai-agents/llms-txt): Publish `/llms.txt`, a short Markdown guide to your site for AI tools. Give the site's name, a one-line summary, and links to the Markdown version of each important page. - [Offer every page as Markdown](https://agentwranglers.se/playbook/website/ready-for-ai-agents/markdown-source-endpoints): Serve every page as Markdown too. Use its address plus `.md`, and answer requests with `Accept: text/markdown`. Point back to the HTML as the canonical copy. - [Decide what each AI crawler may do](https://agentwranglers.se/playbook/website/ready-for-ai-agents/robots-for-ai-crawlers): Decide what each AI crawler may do. Write it in robots.txt as its own user-agent group. - [Publish each page's data as a file](https://agentwranglers.se/playbook/website/ready-for-ai-agents/structured-data-for-agents): Publish each page's JSON-LD (structured facts) in the page and as its own file. Then agents can read the facts without parsing HTML. - [Offer content machines can read](https://agentwranglers.se/playbook/website/ready-for-ai-agents/machine-readable-formats): Offer the site's content as a sitemap, feeds, Markdown and JSON-LD. - [Advertise agent files in headers](https://agentwranglers.se/playbook/website/ready-for-ai-agents/link-headers): List your discovery files in HTTP `Link` headers on every response. These include llms.txt, the sitemap, feeds, the API catalog and agent cards. ## Optional - [Offer all content in one file](https://agentwranglers.se/playbook/website/ready-for-ai-agents/llms-full-txt): Publish `/llms-full.txt` with the full text of every public page in one Markdown file. - [Say how your content may be used](https://agentwranglers.se/playbook/website/ready-for-ai-agents/content-signals): Use `Content-Signal` lines in robots.txt to say whether your content may be used for search, AI answers or AI training. - [State your text-mining choice](https://agentwranglers.se/playbook/website/ready-for-ai-agents/tdmrep): State whether you reserve text-and-data-mining rights. Use a `tdm-reservation` header and `/.well-known/tdmrep.json`, matching robots.txt. - [Sign your bot's requests](https://agentwranglers.se/playbook/website/ready-for-ai-agents/web-bot-auth): If you run a crawler or agent, sign its requests with Web Bot Auth. As a site, don't block unsigned bots by default. - [Explain how agents sign up](https://agentwranglers.se/playbook/website/ready-for-ai-agents/auth-md): If agents must register before using your service, explain how in an `auth.md` file at the root of your service. - [Describe your MCP tools honestly](https://agentwranglers.se/playbook/website/ready-for-ai-agents/mcp-and-tool-discovery): If you run an MCP server, a way for AI assistants to use your tools, publish a server card and label every tool honestly. - [Publish an agent card](https://agentwranglers.se/playbook/website/ready-for-ai-agents/a2a-agent-cards): If you run an A2A agent, one that talks to other agents, publish its card at `/.well-known/agent-card.json`. Keep a working endpoint behind it. - [List skills agents can learn](https://agentwranglers.se/playbook/website/ready-for-ai-agents/agent-skills-discovery): Publish agent skills as `SKILL.md` files, listed in `/.well-known/agent-skills/index.json`. Give each one a digest, a hash of its exact bytes. - [List agent endpoints in DNS](https://agentwranglers.se/playbook/website/ready-for-ai-agents/dns-aid): List your agent endpoints in DNS with SVCB records, as the DNS-AID draft describes. Sign the zone with DNSSEC so answers can be trusted. - [Keep one catalogue of agent tools](https://agentwranglers.se/playbook/website/ready-for-ai-agents/agentic-resource-discovery): List your MCP servers, agents and skills in `/.well-known/ai-catalog.json`. Link it from every page. - [Answer questions from your own pages](https://agentwranglers.se/playbook/website/ready-for-ai-agents/nlweb): Offer an endpoint that answers plain-language questions from your own content, with sources. - [Give in-browser agents your tools](https://agentwranglers.se/playbook/website/ready-for-ai-agents/webmcp): Register the site's tools with the browser through WebMCP. Mark the ones with effects, and report expected failures in the tool's answer. - [Package your knowledge for agents](https://agentwranglers.se/playbook/website/ready-for-ai-agents/okf-bundle): Publish the site's knowledge as an Open Knowledge Format bundle. It has an index, one typed Markdown file per concept, and a change log. - [Index every page's data file](https://agentwranglers.se/playbook/website/ready-for-ai-agents/schemamap): List every JSON-LD file with its types in `/schemamap.xml`. Link it from the page head. - [Ask agents to credit you](https://agentwranglers.se/playbook/website/ready-for-ai-agents/ask-for-credit): If agents may reuse your content in their own work, tell them how to credit it, on every page and in the Markdown copy. ## Avoid - [Show agents the same page as people](https://agentwranglers.se/playbook/website/ready-for-ai-agents/same-content-for-agents): Never serve AI crawlers or agents different content from what people see, chosen by user agent, IP address or bot signature. - Previous: [Standard addresses](https://agentwranglers.se/playbook/website/standard-addresses) - Next: [Privacy](https://agentwranglers.se/playbook/website/privacy) - [All chapters](https://agentwranglers.se/playbook/website)