Agent Wranglers

AGT-59Recommended

Offer content machines can read

Offer the site's content as a sitemap, feeds, Markdown and JSON-LD.

By Agent Wranglers. Updated .

How to do it

Build the sitemap, feeds, Markdown and JSON-LD from one page list, so they always agree. Keep every format stating the same facts, as Showing agents a different page than people says. An agent pays for every byte it reads, so let it take only what it needs from a large data file. Accept filters and a field list in the query string, such as ?level=Required&fields=name,url. Answer an unknown filter value with a 400 that lists the valid ones. Give each record a stable ID and a revision that changes with its content. Then an agent that cached a record can tell when it changed. When a filter also keeps items that have no value for it, say so in the answer. Otherwise an agent reads those items as matches.

Examples

JavaScript example

const pages = [{ path: '/', title: 'Home' }, { path: '/services', title: 'Services' }];
const sitemap = pages.map(p => `<url><loc>https://example.com${p.path}</loc></url>`).join('');
const llms = pages.map(p => `- [${p.title}](https://example.com${p.path}.md)`).join('\n');

Python example

pages = [{"path": "/", "title": "Home"}, {"path": "/services", "title": "Services"}]
sitemap = "".join(f"<url><loc>https://example.com{p['path']}</loc></url>" for p in pages)
llms = "\n".join(f"- [{p['title']}](https://example.com{p['path']}.md)" for p in pages)

How to check

A test reads the page list. Every public page appears in the sitemap and has a Markdown copy and JSON-LD in its HTML, and every feed entry is a listed page. Any data file over 100 KB accepts a filter that cuts the answer below 100 KB. Every record has an ID that no other record shares.

Sources

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/agents-as-users/ready-for-ai-agents/machine-readable-formats">Agent Wranglers agents as users playbook</a>. In code you write, add a comment next to it with the same link.