---
title: "Offer every page as Markdown · Agent Wranglers"
url: https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/markdown-source-endpoints
id: AGT-49
level: Recommended
updated: 2026-10-27
build: a2a6bdfed830
tokens: 1021
licence: "CC BY 4.0"
licence_url: https://creativecommons.org/licenses/by/4.0/
---

# Offer every page as Markdown

For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [Ready for AI agents](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents.md).

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers.md). Updated 2026-10-27.

- ID: AGT-49
- Level: Recommended
- Chapter: [Ready for AI agents](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents.md)
- Rule: Serve every page as Markdown too. Use its address plus `.md`, and answer requests with `Accept: text/markdown`. Point back to the HTML as the canonical copy.
- Check: `curl -sI -H 'Accept: text/markdown' URL` shows `Content-Type: text/markdown`, `Vary: Accept` and the canonical `Link`. A plain browser request to the same URL gets HTML with `Vary: Accept`.
- How: Make a Markdown copy of each page at build time. Serve it at `/page.md`. On the page URL, serve it when `Accept` ranks `text/markdown` above `text/html`, and serve HTML otherwise. Send `Vary: Accept` on both answers. Give the copy front matter (title, URL, updated) and a `Link: <page>; rel="canonical"` header. Add `X-Robots-Tag: noindex` to the Markdown copy, so search engines index only the HTML. Answering `Accept: text/markdown` is a convention some CDNs and agents follow, not a standard.

## Examples

**Using these examples:** they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

### HTTP example

```http
GET /services HTTP/1.1
Host: example.com
Accept: text/markdown

HTTP/1.1 200 OK
Content-Type: text/markdown; charset=utf-8
Vary: Accept
Link: <https://example.com/services>; rel="canonical"
X-Robots-Tag: noindex

---
title: Services
url: https://example.com/services
updated: 2026-10-01
---
# Services
```

### JavaScript example

```js
function prefersMarkdown(accept = '') {
  const weightOf = type => {
    const part = accept.split(',').map(p => p.trim()).find(p => p.split(';')[0] === type);
    if (!part) return 0;
    const q = /;\s*q=([\d.]+)/.exec(part);
    return q ? Number(q[1]) : 1;
  };
  return weightOf('text/markdown') > weightOf('text/html');
}
```

### Python example

```python
def prefers_markdown(accept: str = "") -> bool:
    def weight_of(media_type: str) -> float:
        for part in (p.strip() for p in accept.split(",")):
            name, *params = [x.strip() for x in part.split(";")]
            if name == media_type:
                q = next((x[2:] for x in params if x.startswith("q=")), "1")
                return float(q)
        return 0.0
    return weight_of("text/markdown") > weight_of("text/html")
```

## Sources

- [RFC 7763 (The text/markdown Media Type)](https://www.rfc-editor.org/rfc/rfc7763)
- [RFC 9110 (Vary)](https://www.rfc-editor.org/rfc/rfc9110#field.vary)

## Related rules

- AGT-50 [Link Markdown copies to Markdown copies](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/markdown-links-to-markdown.md) in Agents as users
- APS-34 [Send a hash of the body](https://agentwranglers.se/playbook/application-security/browser-security-headers/digest-fields.md) in Application security

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/markdown-source-endpoints">Agent Wranglers agents as users playbook</a>`. In code you write, add a comment next to it with the same link.

- Previous: [Split large agent content into named parts behind an index](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/named-parts-index.md)
- Next: [Link Markdown copies to Markdown copies](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/markdown-links-to-markdown.md)
- [Ready for AI agents](https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents.md)