AI features and agents

These are the security controls for AI features and agents you ship. Product rules for them, such as approvals, run limits, spend caps and reviewed tools, live in the AI features playbook. Your product as a tool server for outside agents is in the Agents as users playbook, and coding agents in the AI-assisted development playbook.

Required

• Break the lethal trifecta

  APS-84

  Never give one agent all three of private data, untrusted content and a way to send data out. Remove at least one.

• Give agent tools the least power they need

  APS-85

  Give the model only the tools its task needs, each with the narrowest rights, acting as the user and never as an administrator.

• Run code the model writes in a sandbox

  APS-91

  Run code or commands a model writes only in a throwaway sandbox. It has no secrets, no network and fixed limits on time, memory and disk.

Recommended

• Resist direct prompt injection

  APS-82

  Assume any user can talk your model into ignoring its instructions. Enforce every limit in code outside the model, never only in the prompt.

• Resist indirect prompt injection

  APS-83

  Treat every page, file, email and tool result the model reads as data that may contain hostile instructions.

• Treat model output as untrusted

  APS-86

  Handle every model answer like text from a stranger. Escape it before showing it, and never run it as code, SQL or a shell command.

• Keep secrets out of the system prompt

  APS-87

  Write the system prompt as if it will be published. It holds no keys, passwords, internal addresses or private rules.

• Protect agent memory

  APS-88

  Save to long-term memory only what the user said or approved, and keep each user's memory apart.

• Send models only the private data they need

  APS-89

  Send a model provider only the personal data a task needs, under terms that forbid training on it.

• Keep a switch that turns the agent off

  APS-90

  Be able to switch off an agent, or one of its tools, for every user at once and without a deploy.

• Lock local agent servers to your own machine

  APS-92

  A local MCP server or other agent tool server listens on 127.0.0.1 only. It refuses requests with a foreign Origin header and asks for a token.

Read this with

• AI features playbook

  Enforce limits outside the model, break the lethal trifecta and sandbox code the model writes.