---
title: "Run pipeline steps by their dependencies · Agent Wranglers"
url: https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture/pipeline-dependencies
id: DEN-13
level: Recommended
updated: 2026-10-22
tokens: 621
licence: "CC BY 4.0"
licence_url: https://creativecommons.org/licenses/by/4.0/
---

# Run pipeline steps by their dependencies

For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [Loading and change data capture](https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture.md).

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers.md). Updated 2026-10-22.

- ID: DEN-13
- Level: Recommended
- Chapter: [Loading and change data capture](https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture.md)
- Rule: Start each step when the steps it reads from have finished, not at a fixed clock time. Retry a failed step at most 3 times, then stop the steps after it and alert.
- Check: Make one load step fail on staging. No step that reads from it runs, and an alert reaches the owner within 15 minutes. Pause the scheduler past one run, and the heartbeat alert fires.
- How: A clock schedule breaks when one step runs long, and the next step builds on half-loaded data. Use an orchestrator such as Airflow or Dagster, or the dependency graph of the transformation tool. Space retries out with backoff. Watch for runs that never start, as [Alert when a scheduled job stops running](https://agentwranglers.se/playbook/operations/alerts-and-on-call/job-heartbeats.md) says.

## Sources

- [Apache Airflow Tasks](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/tasks.html)

## Related rules

- OPS-23 [Alert when a scheduled job stops running](https://agentwranglers.se/playbook/operations/alerts-and-on-call/job-heartbeats.md) in Operations and observability
- DEN-16 [Alert when data arrives late](https://agentwranglers.se/playbook/data-engineering/data-quality-tests/freshness-limits.md) in Data engineering

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture/pipeline-dependencies">Agent Wranglers data engineering playbook</a>`. In code you write, add a comment next to it with the same link.

- Previous: [Treat each source schema as a contract](https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture/source-schema-contracts.md)
- [Loading and change data capture](https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture.md)