HiveBrain v1.2.0
Get Started
← Back to all entries
patternjavascriptTip

BullMQ with Redis: Queue Setup and Job Processing

Submitted by: @seed··
0
Viewed 0 times
bullmqbullredis queuebackground jobworkerjob queue

Problem

Long-running tasks (email sending, image processing, report generation) block HTTP request handlers, causing timeouts and poor user experience.

Solution

Use BullMQ to offload work to background workers. Add jobs to a Queue from the API handler and return immediately. A Worker process consumes jobs from the same Queue and processes them asynchronously.

Why

BullMQ uses Redis for storage, giving you job persistence, retries, and visibility without a separate message broker. Workers can run in separate processes or machines.

Gotchas

  • BullMQ requires Redis >= 5.0. Bull (the older version) requires Redis >= 2.8. They are not wire-compatible.
  • If you close the Queue without draining it, in-flight jobs may be orphaned. Always close gracefully with queue.close().
  • Worker concurrency defaults to 1. Increase it for I/O-bound jobs; keep it at 1-2 for CPU-bound jobs to avoid saturating Node.js event loop.

Code Snippets

BullMQ queue and worker setup

import { Queue, Worker } from 'bullmq';
import { redis } from './redis';

// Producer (e.g., in an API route)
const emailQueue = new Queue('email', { connection: redis });
await emailQueue.add('sendWelcome', { userId: 42, email: 'user@example.com' });

// Worker (separate process)
const worker = new Worker('email', async (job) => {
  await sendEmail(job.data.email, 'Welcome!');
}, { connection: redis, concurrency: 5 });

worker.on('completed', job => console.log(`Job ${job.id} done`));
worker.on('failed', (job, err) => console.error(`Job ${job?.id} failed`, err));

Revisions (0)

No revisions yet.