Agent Wranglers

ANA-18Avoid

Testing prices or targeting vulnerable people in experiments

Never show two people different prices for the same thing in an A/B test. Leave minors and customers flagged as vulnerable out of every experiment.

How to do it

A random price test charges some customers more for the same product, and they find out by comparing screens. In the EU, a price set for one person by automated means must be disclosed. See Tell customers when a price is personalised for them requires. Test price pages, plan names or bundles instead, with the same price for everyone at a given time. Minors and people in a vulnerable situation are easier to push, and consumer law judges a practice by its effect on them. Read the flag from Flag customers who say they are in a vulnerable situation and the account's age group, and exclude those accounts at assignment. Reviews and endorsements are never tested or written for a test, as Never fake, buy or hide reviews says.

Examples

JavaScript example

export function eligibleForExperiments(account) {
  return !account.isVulnerable && account.ageGroup !== 'minor';
}

export function assertNoPriceVariant(experiment) {
  if (experiment.variants.some(variant => 'price' in variant.changes)) throw new Error(`${experiment.name} changes a price`);
}

Python example

def eligible_for_experiments(account) -> bool:
    return not account.is_vulnerable and account.age_group != "minor"

def assert_no_price_variant(experiment) -> None:
    if any("price" in variant.changes for variant in experiment.variants):
        raise ValueError(f"{experiment.name} changes a price")

How to check

A test of the assignment code finds no experiment whose variants change a price field. An account flagged as vulnerable or under 18 always gets the control in every running experiment.

Sources

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/analytics/experiments/no-price-or-vulnerable-tests">Agent Wranglers product analytics and experiments playbook</a>. In code you write, add a comment next to it with the same link.