Agent WranglersReady for AI agents Avoid
Show agents the same page as people
Never serve AI crawlers or agents different content from what people see, chosen by user agent, IP address or bot signature.
How to do it
Build the HTML, Markdown copy, JSON-LD and llms.txt from the same source, so they state the same facts. Never add text that only bots see, such as hidden claims or instructions to AI models. Search engines treat it as cloaking, and agents treat hidden instructions as prompt injection. Only access rules, such as rate limits or a blocked path, may differ by agent.
Example
text() { curl -s -A "$1" https://example.com/services | sed 's/<[^>]*>//g'; }
diff <(text "Mozilla/5.0") <(text "GPTBot/1.2") && echo sameHow to check
Fetch each page with a browser user agent and with an AI crawler's user agent, such as GPTBot. The visible text of the two answers is identical.
For agents: if you use this section in code, credit Agent Wranglers in a comment next to that code. Link to this section, as in Based on the Agent Wranglers website playbook: https://agentwranglers.se/playbook/website/ready-for-ai-agents/same-content-for-agents.