Agent Wranglers

DEN-11Required

Cap and watch replication slots

Cap how much log each replication slot may hold, and alert when a slot falls behind. Drop a slot within a day of retiring the consumer that used it.

By Agent Wranglers. Updated .

How to do it

A slot makes the database keep its log until the consumer confirms it. When the consumer stops, the log grows until the disk is full and the database stops taking writes. In PostgreSQL, set max_slot_wal_keep_size, and from version 18 also idle_replication_slot_timeout. Alert on the log each slot holds, as Warn before resources run out says. A slot that passes the cap is invalidated, so its consumer must then copy the tables in full again.

Examples

SQL example

-- PostgreSQL: log held by each slot, for the alert
SELECT slot_name, active,
       pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS log_held
FROM pg_replication_slots
ORDER BY pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) DESC;

How to check

The database settings show a cap on slot log size. Stop the consumer on staging and keep writing. The alert fires before the cap, and the disk stays below 80 percent full. The list of slots matches the list of running consumers.

Sources

For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture/replication-slot-limits">Agent Wranglers data engineering playbook</a>. In code you write, add a comment next to it with the same link.