Agent WranglersWarehouse and models
Keep analytics off the production database. Land raw data unchanged, build every table in reviewed code, give each an owner, and cap query costs.
- Recommended 6
Recommended
Keep analytics queries off the production database
DEN-1Run reports, dashboards and analysis against a warehouse or a read replica, never against the primary database the app writes to.
Land raw data unchanged and build tables in layers
DEN-2Load source data into a raw layer exactly as it arrived, then build cleaned and business tables from it in code. Only the loader writes to the raw layer.
Build every warehouse table from reviewed code
DEN-3Write every table the warehouse builds as SQL or code in version control, reviewed and deployed by CI. No shared table is built from a saved query in a BI tool or a notebook.
Give every table an owner and a description
DEN-4Record an owner team, a one-line description and a refresh schedule for every shared table, in a metadata file next to its code.
Build and test models in separate schemas
DEN-5Build models during development and in CI in a personal or per-branch schema, with masked data. Only the deploy job writes to the production schemas.
Cap what each query and each month can cost
DEN-6Set a maximum of bytes scanned or credits per query, and a monthly budget with alerts for the warehouse. Partition large tables by date and require a date filter on them.
For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/data-engineering/warehouse-and-models">Agent Wranglers data engineering playbook</a>. In code you write, add a comment next to it with the same link.