EVT-28Recommended
Store each incoming webhook before you act on it
When a provider sends you a webhook, verify its signature, store the raw body and headers, and answer 2xx. Process the stored copy in a background job.
How to do it
A provider waits only seconds for an answer and retries on a timeout, so slow work in the handler makes duplicates. Verify first, as Sign every webhook describes for both sides. Then insert the event under a unique constraint on the provider's event ID, so a retry inserts nothing. A job processes it as Process each message once says. The stored copy lets you replay an event after a bug fix, and shows support what arrived. Keep stored bodies only as long as the retention schedule allows, since they often hold personal data. Payment providers add a sweep for missed events, as Sweep for missed events every hour shows. Email and text providers send delivery events the same way.
Examples
JavaScript example
app.post('/hooks/payments', express.raw({ type: 'application/json' }), async (req, res) => {
if (!isValidWebhook(req.headers, req.body, PAYMENT_WEBHOOK_SECRET)) return res.sendStatus(401);
const event = JSON.parse(req.body);
await db.query(
'INSERT INTO incoming_webhooks (provider, event_id, body, received_at) VALUES ($1, $2, $3, now()) ON CONFLICT DO NOTHING',
['payments', event.id, req.body.toString('utf8')]);
await queue.add('process-webhook', { provider: 'payments', eventId: event.id }, { jobId: `payments:${event.id}` });
res.sendStatus(204);
});
Python example
@app.post("/hooks/payments", status_code=204)
async def payment_webhook(request: Request):
raw = await request.body()
if not is_valid_webhook(request.headers, raw, PAYMENT_WEBHOOK_SECRET):
raise HTTPException(status_code=401)
event = json.loads(raw)
db.execute(
"INSERT INTO incoming_webhooks (provider, event_id, body, received_at) VALUES (%s, %s, %s, now())"
" ON CONFLICT DO NOTHING", ("payments", event["id"], raw.decode()))
queue.enqueue(process_webhook, "payments", event["id"], job_id=f"payments:{event['id']}")
How to check
With the job worker stopped, send a signed test event. The handler answers 2xx within 1 second and one row holds the raw body. Send it again, and still one row exists. Start the worker, and the event is processed once.
Sources
Related rules
- EML-73 Accept bounce and complaint events only when signed in Email
- EVT-25 Sign every webhook in Integration and events
- EVT-11 Process each message once in Integration and events
- CMP-11 Keep one retention schedule in Compliance and governance
- PAY-20 Sweep for missed events every hour in Payments and billing
For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/events/webhooks/store-incoming-webhooks">Agent Wranglers integration and events playbook</a>. In code you write, add a comment next to it with the same link.