Make the site easy for AI agents

How to do it

Start with the basics of server-rendered HTML, structured data, robots rules, llms.txt and Markdown copies. Then add tool endpoints such as MCP if agents can act on your site. Many of the newer formats in this chapter are drafts, so each rule names its status.

Example

# Quick check that each file answers
for p in / /index.md /llms.txt /robots.txt /sitemap.xml; do
  curl -s -o /dev/null -w "%{http_code} $p\n" "https://example.com$p"
done

How to check

Every discovery file answers 200 with the right content type. Then ask an AI assistant about the site, and it finds and quotes the right pages.