---
title: "Process each message once · Agent Wranglers"
url: https://agentwranglers.se/playbook/events/consuming-events/idempotent-consumers
id: EVT-11
level: Required
risk: money
updated: 2026-10-27
build: a2a6bdfed830
tokens: 1674
licence: "CC BY 4.0"
licence_url: https://creativecommons.org/licenses/by/4.0/
---

# Process each message once

For AI agents: the [site map](https://agentwranglers.se/llms.txt) lists everything here, and this page sits under [Consuming events](https://agentwranglers.se/playbook/events/consuming-events.md).

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers.md). Updated 2026-10-27.

- ID: EVT-11
- Level: Required
- Chapter: [Consuming events](https://agentwranglers.se/playbook/events/consuming-events.md)
- Rule: Give every message, job and webhook a unique ID or key, and make each consumer skip a key it has already processed. Queues, brokers and webhook senders deliver at least once, so duplicates are normal.
- Check: Run each job type twice in a test with the same input. The second run changes 0 rows and sends nothing. Deliver the same webhook event twice, and the receiver acts once and answers 2xx both times. Kill a worker after the side effect and before it acknowledges, and let it retry. Exactly one effect exists.
- Risk: Money lost or taken twice
- How: A worker can crash after the work but before it acknowledges the message, and a sender retries when an answer is slow. Take the key from the message ID, or from the job's purpose, such as `invoice:2026-10:cust_481`. Record it under a unique constraint in the same transaction as the work, and skip the message when the key exists. When several consumers share one store, key each row on the consumer's name and the message ID. A consumer that writes to another store keeps its stream position there, as [Commit stream offsets in the same transaction as the data](https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture/exactly-once-sinks.md) shows. Pass the key on to other services as their idempotency key, as [Send the same idempotency key when you retry a write](https://agentwranglers.se/playbook/api-design/using-other-companies-apis/outgoing-idempotency-keys.md) says. Keep processed keys at least as long as the sender may still retry, such as the 24 hours or more of a webhook sender. Webhook receivers answer 2xx within a few seconds and queue the work, so the sender does not time out and retry. HTTP clients that retry a POST use [idempotency keys](https://agentwranglers.se/playbook/api-design/reliability/idempotency-keys.md) instead.

## Examples

**Using these examples:** they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

### JavaScript example

```js
export async function sendInvoice(job) {
  const key = `invoice:${job.data.month}:${job.data.customerId}`;
  await db.tx(async tx => {
    const claimed = await tx.result('INSERT INTO processed_messages (key) VALUES ($1) ON CONFLICT DO NOTHING', [key]);
    if (claimed.rowCount === 0) return;
    const invoice = await createInvoice(tx, job.data);
    await tx.none('INSERT INTO outbox (kind, ref) VALUES ($1, $2)', ['invoice-email', invoice.id]);
  });
}
```

### Python example

```python
def handle_event(event: dict) -> None:
    with conn.transaction():
        claimed = conn.execute(
            "INSERT INTO processed_messages (key) VALUES (%s) ON CONFLICT DO NOTHING", (event["id"],)
        ).rowcount
        if claimed == 0:
            return
        apply_event(conn, event)
```

## Sources

- [Microsoft Azure Architecture Center, Idempotent Consumer pattern](https://learn.microsoft.com/en-us/azure/architecture/patterns/idempotent-consumer)
- [The Twelve-Factor App, Disposability](https://12factor.net/disposability)

## Related rules

- ARC-91 [Run a change across services as a saga with undo steps](https://agentwranglers.se/playbook/architecture/system-architecture/sagas.md) in Clean code and architecture
- API-32 [Accept idempotency keys on POST that creates or charges](https://agentwranglers.se/playbook/api-design/reliability/idempotency-keys.md) in API design
- DAT-68 [Run every background job inside its tenant](https://agentwranglers.se/playbook/data/multi-tenancy/tenant-context-in-jobs.md) in Data and databases
- PAY-18 [Let provider events move the order](https://agentwranglers.se/playbook/payments/payment-events-and-reconciliation/events-drive-fulfilment.md) in Payments and billing
- MOB-5 [Verify every store purchase on your server](https://agentwranglers.se/playbook/mobile-apps/store-review-and-listing/verify-store-purchases.md) in Mobile apps
- DEN-7 [Make every load safe to rerun](https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture/idempotent-loads.md) in Data engineering
- ECM-26 [Buy each shipping label once](https://agentwranglers.se/playbook/ecommerce/shipping-and-delivery/idempotent-labels.md) in E-commerce and fulfilment
- ECM-27 [Update delivery status from carrier events](https://agentwranglers.se/playbook/ecommerce/shipping-and-delivery/carrier-tracking-events.md) in E-commerce and fulfilment
- EVT-2 [Wrap every event in one envelope](https://agentwranglers.se/playbook/events/contracts-and-schemas/event-envelope.md) in Integration and events
- DEN-39 [Commit stream offsets in the same transaction as the data](https://agentwranglers.se/playbook/data-engineering/loading-and-change-data-capture/exactly-once-sinks.md) in Data engineering
- API-76 [Send the same idempotency key when you retry a write](https://agentwranglers.se/playbook/api-design/using-other-companies-apis/outgoing-idempotency-keys.md) in API design
- EVT-14 [Keep events long enough to replay them](https://agentwranglers.se/playbook/events/consuming-events/event-replay.md) in Integration and events
- EVT-15 [Undo failed multi-step work with compensating steps](https://agentwranglers.se/playbook/events/consuming-events/sagas.md) in Integration and events
- EVT-24 [Replay dead letters with a written procedure](https://agentwranglers.se/playbook/events/background-jobs-and-queues/dead-letter-replay.md) in Integration and events
- EVT-28 [Store each incoming webhook before you act on it](https://agentwranglers.se/playbook/events/webhooks/store-incoming-webhooks.md) in Integration and events

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/events/consuming-events/idempotent-consumers">Agent Wranglers integration and events playbook</a>`. In code you write, add a comment next to it with the same link.

- Next: [Don't rely on event order](https://agentwranglers.se/playbook/events/consuming-events/event-order.md)
- [Consuming events](https://agentwranglers.se/playbook/events/consuming-events.md)