Data checklist

Every rule in the data playbook, by chapter, required rules first. Each links to its own page with how we built it and how to check it.

Schema design

• Required Enforce rules with constraints
• Required Choose exact types
• Required Give every row a stable key
• Recommended Name things the same way everywhere
• Recommended Keep one fact in one place
• Recommended Plan how rows are removed
• Recommended Keep each customer's data apart

Queries and indexes

• Required Pass values as parameters
• Required Index the queries you run
• Required Load related rows in one query
• Recommended Read the query plan
• Recommended Select only the columns you use
• Recommended Page through long lists by key

Transactions and concurrency

• Required Wrap multi-step changes in a transaction
• Required Make conditional writes
• Required Make writes safe to repeat
• Recommended Know your isolation level
• Recommended Send side effects through an outbox
• Recommended Set aside messages that keep failing

Migrations

• Required Change the schema only with migrations
• Required Never edit an applied migration
• Required Expand first and contract later
• Required Test migrations on a copy of production data
• Required Fix production data with reviewed scripts
• Recommended Keep migrations from locking tables

Backups and recovery

• Required Encrypt backups
• Required Prove restores work
• Recommended Keep point-in-time recovery for busy databases

Personal data

• Required Store only the personal data you need
• Required Delete old data automatically
• Required Keep personal data out of test copies
• Required Log who reads personal data
• Recommended Encrypt the most sensitive fields

Operations

• Required Set time limits on every query
• Required Watch slow queries
• Required Let routine maintenance run
• Required Give the app a least-privilege database user
• Recommended Pool database connections
• Recommended Watch size and growth

Caches and copies

• Required Name the source of truth
• Required Decide how stale a cache may be
• Recommended Read from replicas only what may lag
• Recommended Keep search indexes in step

Product analytics

• Required Keep a tracking plan in the code
• Required Validate every event before you store it
• Required Track only after consent
• Required Keep personal data out of events
• Required Record business facts on the server
• Required Write the experiment plan before it starts
• Required Assign each user to one variant for good
• Required Delete raw events on a schedule
• Recommended Define each metric once
• Recommended Keep test and bot traffic out of the numbers