Product analytics and experiments checklist

Every rule in the product analytics and experiments playbook, by chapter, required rules first. Each links to its own page with how we built it and how to check it. This is a lot to check. An Expert can check it for you.

Events and metrics

• Recommended ANA-1 Keep a tracking plan in the code
• Recommended ANA-2 Validate every event before you store it
• Recommended ANA-3 Record business facts on the server
• Recommended ANA-4 Keep test and bot traffic out of the numbers
• Recommended ANA-5 Define each metric once
• Recommended ANA-6 Measure how many new accounts reach their first result

Privacy and consent

• Required EU law ANA-7 Track only after consent
• Required ANA-8 Keep personal data out of events
• Required ANA-9 Hide personal data from session recordings
• Recommended ANA-10 Delete raw events on a schedule

Experiments

• Required EU law ANA-17 Name the lawful basis for every experiment
• Recommended ANA-11 Write the experiment plan before it starts
• Recommended ANA-12 Assign each user to one variant for good
• Recommended ANA-13 Stop the analysis on a sample ratio mismatch
• Recommended ANA-14 Decide only at the planned sample size or with a sequential test
• Recommended ANA-15 Stop an experiment when a guardrail metric breaks
• Optional ANA-16 Keep a holdout group to measure the sum of changes
• Avoid ANA-18 Testing prices or targeting vulnerable people in experiments

Related playbooks

• Records and data map

  In the Compliance and governance playbook. Analytics and experiments need a lawful basis, a retention period and a place in the data map.

• Privacy

  In the Website playbook. The consent banner and the privacy signal decide what analytics may record.

• Warehouse and models

  In the Data engineering playbook. Events copied into a warehouse become tested models with one owner each.

• Catalog and prices

  In the E-commerce and fulfilment playbook. A price set for one person must be disclosed, so price tests follow these rules.

Show the badge on your project

When your project passes every Required rule, you can show this badge on your site or in your README. It links to this checklist, so anyone can run the same checks.

<a href="https://agentwranglers.se/playbook/analytics/checklist"><img src="https://agentwranglers.se/badge.svg" alt="Checked against the Agent Wranglers product analytics and experiments playbook" width="214" height="20"></a>

[![Checked against the Agent Wranglers product analytics and experiments playbook](https://agentwranglers.se/badge.svg)](https://agentwranglers.se/playbook/analytics/checklist)