Reliable Queue Pattern

Guarantee at-least-once message delivery using LMOVE to atomically transfer messages to a processing list, enabling recovery if consumers crash before completing work.

Standard queue operations using RPOP are unreliable—if a consumer fetches a message and crashes before processing completes, the message is lost forever. This pattern ensures no messages are lost.

The Problem with Simple Queues

A naive queue implementation:

1. Producer: LPUSH queue "message"
2. Consumer: RPOP queue — message is removed
3. Consumer crashes during processing
4. Message is gone permanently

The Solution: Atomic Transfer

Instead of removing the message, atomically move it to a "processing" list. The message remains in Redis until explicitly deleted after successful processing.

How It Works

The consumer uses LMOVE (or the older RPOPLPUSH) to atomically transfer a message from the main queue to a processing queue:

LMOVE work_queue processing:worker1 RIGHT LEFT

This single command: - Removes the message from the right of work_queue - Adds it to the left of processing:worker1 - Does both atomically—no message loss possible

After successful processing, remove the message from the processing queue:

LREM processing:worker1 1 "message"

The Processing Lifecycle

1. Dequeue: LMOVE work_queue processing:worker1 RIGHT LEFT
2. Process: Application handles the message
3. Acknowledge: LREM processing:worker1 1 "message"

If the worker crashes between steps 1 and 3, the message remains in processing:worker1.

Recovery from Failures

A separate monitor process (sometimes called a "reaper") periodically scans processing queues for stalled messages:

1. Check each processing queue
2. If a message has been there longer than a timeout (e.g., 10 minutes), assume the worker died
3. Move the message back to the main queue: LMOVE processing:worker1 work_queue RIGHT RIGHT

This guarantees "at-least-once" delivery—messages may be processed multiple times but are never lost.

Blocking Variant

For efficient consumption without polling, use the blocking version:

BLMOVE work_queue processing:worker1 RIGHT LEFT 30

This waits up to 30 seconds for a message to arrive. If the queue is empty, the connection blocks rather than returning immediately.

Per-Worker Processing Queues

Each worker should have its own processing queue (e.g., processing:worker1, processing:worker2). This allows the monitor to identify which worker holds which messages and makes recovery straightforward.

Delivery Guarantees

This pattern provides at-least-once delivery:

• Messages are never lost (always exist in some queue)
• Messages may be processed multiple times if a worker crashes after processing but before acknowledgment

For exactly-once semantics, make message handlers idempotent—processing the same message twice should produce the same result as processing it once.

When to Use

• Job queues requiring reliability
• Background task processing
• Any scenario where message loss is unacceptable
• Systems where workers may crash or be killed

When to Consider Alternatives

For more complex requirements (multiple consumers per message, message replay, consumer groups), consider Redis Streams instead, which provide these features natively.

← Back to Index | Markdown source