Product analytics

Required

• Keep a tracking plan in the code

  Required

  List every analytics event in one file in the repository, with its name, purpose, owner and properties. Name events as an object and a past-tense action.

• Validate every event before you store it

  Required

  Check each incoming event against its schema at the collector. Store valid events, and set invalid ones aside with the reason.

• Track only after consent

  Required

  Send no analytics event that uses device storage or identifies a person until that person has agreed. Read the stored choice in the tracking module itself.

• Keep personal data out of events

  Required

  Never put names, email addresses, phone numbers, free text or full URLs with query strings in event properties. Use an internal ID that analytics cannot turn back into a person.

• Record business facts on the server

  Required

  Record events that count money, sign-ups or other business facts on the server, at the moment the fact is committed. Use browser events only for behaviour in the page.

• Write the experiment plan before it starts

  Required

  Before an A/B test starts, write down its one primary metric, guardrail metrics, minimum effect, sample size and stop date. Decide only at that sample size.

• Assign each user to one variant for good

  Required

  Assign variants by hashing a stable user ID with the experiment name, so a person sees the same variant on every visit and device. Log an exposure when the variant is shown.

• Delete raw events on a schedule

  Required

  Keep raw events with an analytics ID for a fixed time only, such as 14 months. Then delete them or roll them up into counts without IDs.

Recommended

• Define each metric once

  Recommended

  Define every business metric once, in version control, and build every dashboard and report from that definition.

• Keep test and bot traffic out of the numbers

  Recommended

  Mark every event from staff, test accounts, automated tests and known bots at the collector, and leave marked events out of every metric.