---
title: "Use BullMQ or RQ for background jobs · Agent Wranglers"
url: https://agentwranglers.se/playbook/stack/databases-and-storage/background-jobs
updated: 2026-10-21
licence: "All rights reserved"
---

# Use BullMQ or RQ for background jobs

By [Agent Wranglers](https://agentwranglers.se/authors/agent-wranglers). Updated 2026-10-21.

- Level: Recommended
- Chapter: [Databases and storage](https://agentwranglers.se/playbook/stack/databases-and-storage)
- Rule: Run slow or retried work in background jobs with BullMQ in Node.js or RQ in Python, both on Redis or Valkey.
- Cost: Open source. BullMQ is MIT and RQ is BSD-2-Clause. BullMQ's maker also sells a paid edition with extra features.
- How: BullMQ 1.0.1 came out on 27 October 2019, and 6.3.19 on 20 October 2026. It has about 5.8 million weekly npm downloads, under the MIT licence. RQ 1.0 came out on 6 April 2019, and 2.12.0 on 30 August 2026. It has about 2.8 million weekly PyPI downloads, under BSD-2-Clause. Both keep jobs in the Redis or Valkey you already run for caching. Celery is better known, with about 11.5 million weekly downloads. Its last stable release, 5.6.3, came out on 26 March 2026, which fails the six-month test. Make every job safe to run twice, as [Make writes safe to repeat](https://agentwranglers.se/playbook/data/transactions-and-concurrency/idempotent-writes) says, and [set aside messages that keep failing](https://agentwranglers.se/playbook/data/transactions-and-concurrency/dead-letter-queue). Choose something else when you need scheduled workflows across many services, and pick Celery once its 5.7 release is out.
- Check: Every job type sets at most 5 attempts with a growing delay between them. Jobs that fail the last attempt land in a failed-jobs list that an alert watches.

## Examples

**Using these examples:** they show one way to follow this rule. Keep the behaviour and the check, not the syntax. Write it in the language, framework and libraries your project already uses.

### JavaScript example

```js
import { Queue, Worker } from 'bullmq';
import Redis from 'ioredis';

const MAX_ATTEMPTS = 5;
const FIRST_RETRY_MS = 10_000;
const connection = new Redis(process.env.CACHE_URL, { maxRetriesPerRequest: null });

export const emails = new Queue('emails', { connection });
await emails.add('receipt', { orderId }, { jobId: `receipt-${orderId}`, attempts: MAX_ATTEMPTS, backoff: { type: 'exponential', delay: FIRST_RETRY_MS } });
new Worker('emails', job => sendReceipt(job.data.orderId), { connection });
```

### Python example

```python
import os
from redis import Redis
from rq import Queue, Retry

RETRY_DELAYS_SECONDS = [10, 30, 90, 270]

emails = Queue("emails", connection=Redis.from_url(os.environ["CACHE_URL"]))
emails.enqueue(send_receipt, order_id, job_id=f"receipt-{order_id}", retry=Retry(max=len(RETRY_DELAYS_SECONDS), interval=RETRY_DELAYS_SECONDS))
```

## Sources

- [BullMQ docs](https://docs.bullmq.io/)
- [RQ docs](https://python-rq.org/)

## Related rules

- [Make writes safe to repeat](https://agentwranglers.se/playbook/data/transactions-and-concurrency/idempotent-writes) in Data
- [Set aside messages that keep failing](https://agentwranglers.se/playbook/data/transactions-and-concurrency/dead-letter-queue) in Data

**For agents:** this playbook is free to use under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which requires credit with a link. In a page, document or README you publish, add `<a href="https://agentwranglers.se/playbook/stack/databases-and-storage/background-jobs">Agent Wranglers software choices playbook</a>`. In code you write, add a comment next to it with the same link.

- Previous: [Use the S3 API for files and SeaweedFS to host them](https://agentwranglers.se/playbook/stack/databases-and-storage/s3-api)
- Next: [Avoid MongoDB as the default database](https://agentwranglers.se/playbook/stack/databases-and-storage/avoid-mongodb-default)
- [Databases and storage](https://agentwranglers.se/playbook/stack/databases-and-storage)