---
title: "Loading and change data capture · Agent Wranglers"
url: https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture
parts: 7
tokens_total: 5576
updated: 2026-10-22
tokens: 919
licence: "CC BY 4.0"
licence_url: https://creativecommons.org/licenses/by/4.0/
---

# Loading and change data capture

For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [Data engineering](https://agentwranglers.se/playbook/data-engineering.md).

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers.md). Updated 2026-10-22.

> Load data so a rerun changes nothing, carry deletes as well as updates, read changes from the log, and stop when a source changes shape.

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture">Agent Wranglers data engineering playbook</a>`. In code you write, add a comment next to it with the same link.

## Required

- DEN-9 [Carry deletes into the warehouse](https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture/propagate-deletes.md) · 693 tokens · updated 2026-10-22
  Copy deletes from every source into the warehouse, along with inserts and updates. Within 24 hours of a delete in the source, the row and every row built from it are gone.
- DEN-11 [Cap and watch replication slots](https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture/replication-slot-limits.md) · 775 tokens · updated 2026-10-22
  Cap how much log each replication slot may hold, and alert when a slot falls behind. Drop a slot within a day of retiring the consumer that used it.

## Recommended

- DEN-7 [Make every load safe to rerun](https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture/idempotent-loads.md) · 983 tokens · updated 2026-10-22
  Write each load so that running it twice for the same window gives the same rows. Merge on the source key or replace whole partitions, never append without a key.
- DEN-8 [Load only what changed since the last run](https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture/incremental-loads.md) · 918 tokens · updated 2026-10-22
  Load only rows changed since the last successful run, by a change column or a log position. Save that position after the load commits, and re-read an overlap of at least 15 minutes.
- DEN-10 [Read changes from the database log](https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture/log-based-cdc.md) · 681 tokens · updated 2026-10-22
  For tables that change often or must reach the warehouse within 15 minutes, read changes from the database's write-ahead log or binlog. Do not poll those tables with queries.
- DEN-12 [Treat each source schema as a contract](https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture/source-schema-contracts.md) · 942 tokens · updated 2026-10-22
  Write down the columns and types the pipeline reads from each source. Stop the load and alert the owner when a source drops, renames or retypes one of them.
- DEN-13 [Run pipeline steps by their dependencies](https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture/pipeline-dependencies.md) · 584 tokens · updated 2026-10-22
  Start each step when the steps it reads from have finished, not at a fixed clock time. Retry a failed step at most 3 times, then stop the steps after it and alert.

- Previous: [Warehouse and models](https://agentwranglers.se/playbook/data-engineering/warehouse-and-models.md)
- Next: [Data quality tests](https://agentwranglers.se/playbook/data-engineering/data-quality-tests.md)
- [All chapters](https://agentwranglers.se/playbook/data-engineering.md)