Adversarial testing playbook

An adversarial testing playbook for any website, API or AI agent. It turns an AI agent into a careful attacker of your own software, with your written permission.

Each rule names a security hole and how to close it. Its check is the attack the agent runs to prove the hole is closed. Start with Rules of engagement, because they keep the test safe and legal.

The rules follow the common lists of security risks, such as the OWASP Top 10 for web apps, APIs and AI models. Headers and HTTPS for websites are in the website security chapter.

Chapters

1. Rules of engagement

   Test only what you own or have written permission to test. Agree the scope, use a copy of the system, and stop when harm is possible.

   • Required 7

2. Injection

   Keep data and code apart. Untrusted input must never become part of a query, a page, a shell command, a template or a header.

   • Required 8

3. Access and accounts

   Check on the server who is asking and what they may do, on every request. Make accounts hard to guess, steal or take over.

   • Required 12
   • Recommended 1

4. Data and secrets

   Keep keys out of code and pages. Show short errors, close debug doors and hidden files, and send back only the data each request needs.

   • Required 11

5. Requests and files

   Stop the server fetching addresses an attacker picks, check every uploaded file, keep file paths inside their folder, and limit redirects and sharing.

   • Required 7

6. Abuse and limits

   Limit how fast and how much anyone can ask for. Give the same answer either way, and check every price, step and count on the server.

   • Required 6
   • Recommended 2

7. Supply chain and setup

   Most attacks walk in through someone else's code or a forgotten setting. Pin what you use, keep it current and give every account only what it needs.

   • Required 8
   • Recommended 2

8. AI agents and models

   Attacks on your own AI features. Prompt injection, leaked data, tools that do too much, poisoned memory and runaway costs, with the test for each.

   • Required 14
   • Recommended 3

9. Hostile agents and bots

   AI agents and bots attack sites at scale. Verify who an agent is, limit scraping, mark what users wrote and keep agent actions reversible.

   • Required 2
   • Recommended 2

All 85 rules on one page