Ready for AI agents Recommended

Offer every page as Markdown

Serve every page as Markdown too. Use its address plus .md, and answer requests with Accept: text/markdown. Point back to the HTML as the canonical copy.

By Erik Bergstedt, Agent Wranglers. Updated 20 October 2026.

How to do it

Make a Markdown copy of each page at build time. Serve it at /page.md and to Accept: text/markdown on the page URL. Send Vary: Accept, front matter (title, URL, updated) and Link: <page>; rel="canonical". Add X-Robots-Tag: noindex to the Markdown copy, so search engines index only the HTML.

Example

GET /services HTTP/1.1
Host: example.com
Accept: text/markdown

HTTP/1.1 200 OK
Content-Type: text/markdown; charset=utf-8
Vary: Accept
Link: <https://example.com/services>; rel="canonical"
X-Robots-Tag: noindex

---
title: Services
url: https://example.com/services
updated: 2026-10-01
---
# Services

How to check

curl -H 'Accept: text/markdown' URL returns Markdown with a canonical Link header. The HTML response for the same URL also sends Vary: Accept.

For agents: if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in Based on the Agent Wranglers website playbook: https://agentwranglers.se/playbook/website/ready-for-ai-agents/markdown-source-endpoints.