Validate every event before you store it

How to do it

Compile the tracking plan into JSON Schema 2020-12 validators, such as Ajv's Ajv2020 class or Python's jsonschema. Reject unknown events, unknown properties and wrong types. Write rejects to a separate table with the error and the app version, as with a dead letter queue. Alert when rejects pass 1 percent of events in an hour, because a release has broken tracking.

Example

import Ajv2020 from 'ajv/dist/2020.js';

const ajv = new Ajv2020({ allErrors: true });
const validators = Object.fromEntries(Object.entries(EVENT_SCHEMAS).map(([name, schema]) => [name, ajv.compile(schema)]));

export async function collect(event) {
  const validate = validators[event.name];
  if (validate?.(event.properties)) return db.events.insert(event);
  return db.eventRejects.insert({ event, error: validate ? ajv.errorsText(validate.errors) : 'unknown event' });
}

from jsonschema import Draft202012Validator

VALIDATORS = {name: Draft202012Validator(schema) for name, schema in EVENT_SCHEMAS.items()}

def collect(event: dict):
    validator = VALIDATORS.get(event["name"])
    errors = [e.message for e in validator.iter_errors(event["properties"])] if validator else ["unknown event"]
    if errors:
        return db.event_rejects.insert(event=event, error="; ".join(errors))
    return db.events.insert(event)

How to check

Send one valid and one invalid event to the staging collector. The valid one is in the events table, and the invalid one is in the rejects table with its error.