Agent WranglersBackground jobs and queues
Move slow work out of requests into queues. Limit each job's time, bound the backlog, keep urgent work first, and set aside and replay what keeps failing.
- Required 1
- Recommended 8
Required
Bound the backlog and push back on producers
EVT-18Set a maximum backlog for every queue, as a count or an age. When it is reached, slow or refuse new work instead of letting the backlog grow without end.
Recommended
Move slow work out of the request
EVT-16Run work the visitor does not need in the answer as a background job. Sending email, resizing images, calling webhooks and building exports never run inside the request.
Give every job a time limit
EVT-17Set a maximum run time on each job type. Keep the queue's lock or visibility time longer than that limit, so no second worker starts a job that still runs.
Keep urgent jobs in their own queue
EVT-19Put jobs a person waits for, such as sign-in codes and password resets, in a separate queue with its own workers. Bulk work, such as exports and newsletters, never delays them.
Put record IDs in jobs instead of copies
EVT-20Store the record's ID and the few inputs the job needs in the job. The worker reads the current record when it runs.
Spread scheduled work across the hour
EVT-21Give each scheduled job its own start minute instead of the full hour. Split jobs that touch every customer into batches with gaps between them.
Run each scheduled job once at a time
EVT-22Take a lock before each scheduled job starts, so a second copy skips its run. Jobs that charge, send or delete must also be safe to run twice.
Set aside messages that keep failing
EVT-23After a set number of failed attempts, move a message or job to a separate dead letter queue and alert. Never retry it forever or drop it.
Replay dead letters with a written procedure
EVT-24Write one procedure for each dead letter queue. It covers how to read the failures, fix the cause, replay the messages at a capped rate, and record what was replayed or dropped.
Read this with
For agents: this playbook is free to use under CC BY 4.0, which requires credit with a link. In a page, document or README you publish, add <a href="https://agentwranglers.se/playbook/events/background-jobs-and-queues">Agent Wranglers integration and events playbook</a>. In code you write, add a comment next to it with the same link.