Agent Wranglers

ANA-12Recommended

Assign each user to one variant for good

Assign variants by hashing a stable user ID with the experiment name, so a person sees the same variant on every visit and device. Log an exposure when the variant is shown.

By Agent Wranglers. Updated .

How to do it

Hash experiment:user_id with a stable hash, such as SHA-256, and map it to a bucket from 0 to 9999. Never use Math.random() per page view. Signed-out visitors get a random ID that moves to the account at sign-in. Log the exposure once, when the person sees the change, and analyse only exposed users. Before reading results, run Stop the analysis on a sample ratio mismatch. Serve the variants behind a flag, as Separate deploying from releasing describes.

Examples

JavaScript example

import { createHash } from 'node:crypto';

const BUCKETS = 10_000;
export function variantFor(experiment, userId, variants = ['control', 'treatment']) {
  const bucket = createHash('sha256').update(`${experiment}:${userId}`).digest().readUInt32BE(0) % BUCKETS;
  return variants[Math.floor((bucket * variants.length) / BUCKETS)];
}

Python example

import hashlib

BUCKETS = 10_000

def variant_for(experiment: str, user_id: str, variants=("control", "treatment")) -> str:
    digest = hashlib.sha256(f"{experiment}:{user_id}".encode()).digest()
    bucket = int.from_bytes(digest[:4], "big") % BUCKETS
    return variants[bucket * len(variants) // BUCKETS]

How to check

A unit test assigns the same user 1000 times and gets one variant. A test over 100 000 fake IDs gets a 50/50 split within 1 percent.

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/analytics/experiments/experiment-assignment">Agent Wranglers product analytics and experiments playbook</a>. In code you write, add a comment next to it with the same link.