Agent Wranglers

Product analytics and experiments playbook

Rules for product analytics and A/B tests. A tracking plan in code, consent before tracking, one definition per metric, and experiments you can trust.

  • Required 4
  • Recommended 12
  • Optional 1
  • Avoid 1

By Agent Wranglers. Updated .

A product analytics and experimentation playbook for any app or website that counts what people do and tests changes on them. It works with a hosted tool or your own events table.

It starts with events and metrics. Every event is planned in code and validated, business facts come from the server, and each metric has one definition. Then it covers privacy, because analytics touches people before anything else does. The last chapter covers A/B tests, from the plan and assignment to the checks that stop a false win.

Numbers that nobody can trust are worse than no numbers, because teams act on them. Each rule has a check that proves the count is right, or that a person is left out when they said no.

Consent for marketing messages lives with each channel, in the notifications and email playbooks. Customer reviews follow the e-commerce playbook. Analytics copied into a warehouse follows the data engineering playbook.

Chapters

  1. Events and metrics

    Plan every event in code and validate it, record business facts on the server, keep test traffic out, and define each metric once.

    • Recommended 6

  2. Privacy and consent

    Track only after a yes, keep personal data out of events and recordings, and delete raw events on a schedule. Marketing consent lives with each channel.

    • Required 3
    • Recommended 1

  3. Experiments

    Plan each test before it starts, assign people for good and check the split. Decide at the planned size, watch guardrails, and leave prices out.

    • Required 1
    • Recommended 5
    • Optional 1
    • Avoid 1

Rules from regional law

Most rules here hold for every project. The rules below come from the law of one region. Follow them when your project serves people there.

European Union (EU)

All 18 rules on one page

This is a lot to check. An Expert can check it for you.

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/analytics">Agent Wranglers product analytics and experiments playbook</a>. In code you write, add a comment next to it with the same link.