Ready for AI agents

Recommended

• Make the site easy for AI agents

  Recommended

  Make the site easy for AI agents to read and use. Put content in the HTML, and add structured data, Markdown copies, clear crawler rules and discovery files.

• Give AI tools a site guide

  Recommended

  Publish /llms.txt, a short Markdown guide to your site for AI tools. Give the site's name, a one-line summary, and links to the Markdown version of each important page.

• Offer every page as Markdown

  Recommended

  Serve every page as Markdown too. Use its address plus .md, and answer requests with Accept: text/markdown. Point back to the HTML as the canonical copy.

• Decide what each AI crawler may do

  Recommended

  Decide what each AI crawler may do. Write it in robots.txt as its own user-agent group.

• Publish each page's data as a file

  Recommended

  Publish each page's JSON-LD (structured facts) in the page and as its own file. Then agents can read the facts without parsing HTML.

• Offer content machines can read

  Recommended

  Offer the site's content as a sitemap, feeds, Markdown and JSON-LD.

• Advertise agent files in headers

  Recommended

  List your discovery files in HTTP Link headers on every response. These include llms.txt, the sitemap, feeds, the API catalog and agent cards.

Optional

• Offer all content in one file

  Optional

  Publish /llms-full.txt with the full text of every public page in one Markdown file.

• Say how your content may be used

  Optional

  Use Content-Signal lines in robots.txt to say whether your content may be used for search, AI answers or AI training.

• State your text-mining choice

  Optional

  State whether you reserve text-and-data-mining rights. Use a tdm-reservation header and /.well-known/tdmrep.json, matching robots.txt.

• Sign your bot's requests

  Optional

  If you run a crawler or agent, sign its requests with Web Bot Auth. As a site, don't block unsigned bots by default.

• Explain how agents sign up

  Optional

  If agents must register before using your service, explain how in an auth.md file at the root of your service.

• Describe your MCP tools honestly

  Optional

  If you run an MCP server, a way for AI assistants to use your tools, publish a server card and label every tool honestly.

• Publish an agent card

  Optional

  If you run an A2A agent, one that talks to other agents, publish its card at /.well-known/agent-card.json. Keep a working endpoint behind it.

• List skills agents can learn

  Optional

  Publish agent skills as SKILL.md files, listed in /.well-known/agent-skills/index.json. Give each one a digest, a hash of its exact bytes.

• List agent endpoints in DNS

  Optional

  List your agent endpoints in DNS with SVCB records, as the DNS-AID draft describes. Sign the zone with DNSSEC so answers can be trusted.

• Keep one catalogue of agent tools

  Optional

  List your MCP servers, agents and skills in /.well-known/ai-catalog.json. Link it from every page.

• Answer questions from your own pages

  Optional

  Offer an endpoint that answers plain-language questions from your own content, with sources.

• Give in-browser agents your tools

  Optional

  Register the site's tools with the browser through WebMCP. Mark the ones with effects, and report expected failures in the tool's answer.

• Package your knowledge for agents

  Optional

  Publish the site's knowledge as an Open Knowledge Format bundle. It has an index, one typed Markdown file per concept, and a change log.

• Index every page's data file

  Optional

  List every JSON-LD file with its types in /schemamap.xml. Link it from the page head.

• Ask agents to credit you

  Optional

  If agents may reuse your content in their own work, tell them how to credit it, on every page and in the Markdown copy.

Avoid

• Show agents the same page as people

  Avoid

  Never serve AI crawlers or agents different content from what people see, chosen by user agent, IP address or bot signature.