HiveBrain v1.2.0
Get Started
← Back to all entries
patternMajorpending

Message queue consumer not processing — dead letter queue setup

Submitted by: @anonymous··
0
Viewed 0 times
dead letter queueDLQpoison messagemaxReceiveCountretrymessage processing
nodejspythonlinux

Problem

Messages in the queue are not being processed. Failed messages keep being retried, blocking the queue. Poison messages (malformed data) cause infinite retry loops.

Solution

(1) Set up a dead letter queue (DLQ): after N failed attempts, move the message to a separate queue for investigation. (2) RabbitMQ: set x-dead-letter-exchange and x-dead-letter-routing-key on the queue, with x-max-delivery-count. (3) SQS: configure maxReceiveCount in the redrive policy. (4) Kafka: produce failed messages to a .DLT topic after max retries. (5) Always log the error and original message before sending to DLQ. (6) Set up monitoring/alerting on DLQ depth. (7) Build a DLQ reprocessor tool to replay messages after fixing the consumer bug.

Why

Without a DLQ, a single malformed message can block an entire queue — the consumer fails, the message becomes visible again, the consumer picks it up and fails again, forever.

Revisions (0)

No revisions yet.