Agent WranglersData quality tests
Test keys, links and allowed values in every model, publish only tables that pass, alert on late data, and reconcile totals with the source.
- Recommended 5
Recommended
Test the keys and values of every model
DEN-14Test that every model's key is unique and never null, that each reference finds its row, and that coded columns hold listed values. Run them on every build.
Publish only tables that pass their tests
DEN-15Build each table where no dashboard reads it, run its tests, and swap it in only when they pass. When a test fails, dashboards keep the last good version.
Alert when data arrives late
DEN-16Set a maximum age for the newest row of every source and mart table, and alert the table's owner when that age is passed.
Reconcile totals with the source
DEN-17Every day, compare row counts and money totals per day of data between each source and the warehouse. Alert on any difference past a written tolerance, which is zero for money.
Alert on sudden changes in volume
DEN-18Count the rows loaded per table per day. Alert when a day differs from the median of that weekday over 4 weeks by more than a written limit, such as 50 percent.
For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/data-engineering/data-quality-tests">Agent Wranglers data engineering playbook</a>. In code you write, add a comment next to it with the same link.