Keep events long enough to replay them

How to do it

A consumer with a bug writes wrong data for a day. With the events still in the log, you fix the code and rewind its position to before the bug, and it processes them again. That needs retention longer than the time to notice and fix, such as 7 days. The consumer must also be safe to rerun, as Process each message once requires. Kafka can reset a group's offsets to a date. A compacted topic keeps only the last event per key, so it serves as current state, not as history. Events that hold personal data need a retention that matches Keep one retention schedule, so keep IDs in events rather than personal fields.

Examples

Shell example

# Rewind one consumer group to a point in time on staging
kafka-consumer-groups.sh --bootstrap-server kafka.staging.example.com:9092 \
  --group order-emails --topic shop.orders \
  --reset-offsets --to-datetime 2026-10-26T09:00:00.000 --execute

Python example

from confluent_kafka import Consumer, TopicPartition

def rewind_to(consumer: Consumer, topic: str, partitions: list[int], timestamp_ms: int) -> None:
    wanted = [TopicPartition(topic, p, timestamp_ms) for p in partitions]
    consumer.assign(consumer.offsets_for_times(wanted, timeout=10))

JavaScript example

export async function rewindTo(admin, groupId, topic, timestampMs) {
  const offsets = await admin.fetchTopicOffsetsByTimestamp(topic, timestampMs);
  await admin.setOffsets({ groupId, topic, partitions: offsets.map(({ partition, offset }) => ({ partition, offset })) });
}

How to check

Every topic's retention setting appears in the written list, each at least as long as the stated recovery window. On staging, reset a test consumer to 24 hours ago. It processes the events again and the target holds no duplicates.

Sources

• Apache Kafka design, consumers that rewind

Related rules

• EVT-11 Process each message once in Integration and events
• CMP-11 Keep one retention schedule in Compliance and governance