Redis Message Queue: Can Redis Replace a Real Broker?

Yes. A Redis message queue is a real option, and there are three ways to build one: Lists for simple work distribution, Pub/Sub for fire-and-forget broadcast that stores nothing, and Streams for consumer groups with acknowledgements and at-least-once delivery. Streams is the only one that behaves like an actual broker. What you give up in every case is routing, per-message TTL, priorities and dead-lettering — features you will end up writing yourself.

The question is rarely whether Redis can do it. It can. The question is what happens the first time something fails.

Yes, and this is the oldest Redis message queue pattern — still the right answer for a lot of background job processing.

A producer pushes with LPUSH. A worker blocks on BRPOP, which waits for an item instead of polling. Multiple workers on the same key compete, and each item goes to exactly one of them. That is a work queue in two commands, on infrastructure you already run.

LPUSH jobs:email '{"to":"user@example.com","template":"welcome"}'
BRPOP jobs:email 0

The failure mode is the obvious one. BRPOP removes the item and hands it to the worker. If that worker dies mid-processing, the item is gone — nothing knows it was in flight and nothing will redeliver it.

The mitigation is the reliable-queue pattern: move the item atomically to a per-worker processing list with BLMOVE (or the older BRPOPLPUSH) instead of popping it, then delete it from that list once the work is done. Items left behind in a processing list are recoverable.

That works, and plenty of production systems run on it. Note what you have built: an acknowledgement mechanism, a reaper to detect abandoned processing lists, and a retry policy. Those are broker features, now living in your application, maintained by your team, and tested only as well as you test them.

No, and this is the mechanism most often mistaken for a Redis message queue while being the one least suited to the job.

SUBSCRIBE registers interest in a channel, PUBLISH sends to everyone currently subscribed. Redis's documentation is unambiguous about what that means:

Redis' Pub/Sub exhibits at-most-once message delivery semantics. As the name suggests, it means that a message will be delivered once if at all. Once the message is sent by the Redis server, there's no chance of it being sent again. If the subscriber is unable to handle the message (for example, due to an error or a network disconnect) the message is forever lost.

Nothing is stored. A subscriber that is restarting, deploying, briefly network-partitioned or just slow misses everything published in that window, permanently, with no signal that it happened.

That makes Pub/Sub excellent for a narrow set of jobs: cache invalidation, live dashboard updates, presence notifications — anything where the next message makes the missed one irrelevant. It makes it wrong for order processing, payments, or anything a person will later ask you to account for.

Two useful details. Pub/Sub is unrelated to the keyspace, so a message published on database 10 reaches a subscriber on database 1; prefix your channels if you need environment scoping. And in cluster deployments, sharded Pub/Sub (SSUBSCRIBE, SPUBLISH) confines messages to a single shard rather than propagating them across the cluster bus, which is how you scale it horizontally.

This is as close as it gets. Streams is where Redis stops being a data structure you can abuse into a queue and starts being something you can reasonably run a work pipeline on.

XADD appends an entry and returns a time-ordered ID in <milliseconds>-<sequence> form. Entries persist in the keyspace. XREAD can block for new entries. Consumers can read historical ranges with XRANGE.

Consumer groups are the real feature:

XGROUP CREATE events mygroup $
XREADGROUP GROUP mygroup consumer1 STREAMS events >
XACK events mygroup 1692632086370-0

Each entry goes to one consumer within a group, and multiple groups can consume the same stream independently. A delivered entry sits in that group's Pending Entries List until it is acknowledged with XACK. Inspect the backlog with XPENDING. If a consumer dies with entries outstanding, another can take ownership with XCLAIM or XAUTOCLAIM and finish the work.

That combination gives at-least-once delivery: nothing is lost when a consumer fails, but an entry processed just before a crash can be redelivered. Your consumers need to be idempotent. This is the same requirement any at-least-once broker imposes — it is not a Redis shortcoming, just one people forget until they see duplicate charges.

Streams grow forever unless you cap them. XADD accepts MAXLEN, and XTRIM handles it after the fact; the ~ form trims approximately and is meaningfully cheaper than exact trimming. MINID trims by entry ID rather than count, which suits time-based retention. Pick one at design time. An unbounded stream in an in-memory store is a memory exhaustion incident with a delay fuse.

Redis Streams closes the delivery-guarantee gap. It does not close the others.

Routing. RabbitMQ puts an exchange between publishers and queues. Direct, topic, fanout and headers exchanges decide where a message lands based on routing keys and attributes, as configuration. Redis has no equivalent — a publisher writes to a named key. Any content-based routing is code you write, and it becomes a second thing to keep in sync with your topology.

Per-message TTL. RabbitMQ supports TTL per message via the expiration property and per queue, taking the lower of the two when both are set. It even documents the sharp edge honestly: "Only when expired messages reach the head of a queue will they actually be discarded." Redis TTLs apply to whole keys, not to individual list items or stream entries. Expiring one message and not its neighbours is not a thing Redis does.

Priorities. RabbitMQ supports priority queues. Redis has no priority concept for list items or stream entries. Simulating it means multiple keys and workers that check them in order, which reintroduces starvation problems that brokers already solved.

Dead-lettering. RabbitMQ dead-letters automatically on rejection, TTL expiry, queue length limit, or exceeding a quorum queue's delivery limit. Redis Streams gives you the raw material — XPENDING shows what is stuck and delivery counts are tracked — but routing poison messages to a quarantine stream, capping retries and alerting on it is entirely your code.

Delivery counting and retry policy. Related, and worth separating: a broker gives you a delivery limit as a setting. In Redis you read the counter and decide what to do about it, everywhere, consistently, forever.

None of this makes Redis the wrong choice. It makes the choice a trade: you are accepting ownership of features that come free elsewhere, in exchange for not running another system.

Redis is an in-memory store that writes to disk. How much you lose in a crash is a configuration decision, and it is worth knowing the numbers Redis itself publishes.

RDB takes point-in-time snapshots at configured save points. Redis's documentation is direct about the exposure: "you'll usually create an RDB snapshot every five minutes or more, so in case of Redis stopping working without a correct shutdown for any reason you should be prepared to lose the latest minutes of data." Snapshotting also forks the process, and on a large dataset that fork can stall the server briefly — a real operational effect we cover alongside the other failure modes in our Redis high availability work.

AOF logs every write command, with three fsync policies. appendfsync always is safest and slowest. appendfsync everysec is the suggested default, and Redis states you "may lose 1 second of data if there is a disaster." appendfsync no leaves it to the kernel.

Redis's guidance is to run both if you want durability comparable to a relational database. Two consequences for queueing:

1. A message acknowledged to a producer is not necessarily on disk. With everysec there is up to a second of accepted-but-unwritten work at any moment.
2. Replication is asynchronous. A primary failure can lose writes that were acknowledged but not yet replicated — the same trade Redis makes everywhere for speed.

If your requirement is "this message cannot be lost," Redis can be configured close to that, but you are paying in latency and you should verify the configuration rather than assume it.

Use Redis for queueing when:

• Redis is already in your architecture and adding a broker means adding an operational commitment
• The topology is simple — producers to one pool of workers, no content-based routing
• Latency matters more than delivery guarantees, and re-running a lost job is acceptable
• Volume is high and messages are short-lived, which is what an in-memory store is built for
• The team is comfortable owning acknowledgement, retry and dead-letter logic in application code

Use RabbitMQ when:

• Routing is non-trivial and you would rather configure it than code it
• You need per-message TTL, priorities or dead-lettering as broker features
• Delivery guarantees appear in a compliance document or a customer contract
• Different consumers need different subsets of the same publisher's messages
• You want operational tooling — management UI, per-queue metrics, policies — without building it

These are not mutually exclusive, and in practice the strongest architectures use both: Redis for caching, rate limiting, ephemeral state and fast in-process fan-out, RabbitMQ for messages with guarantees attached. We wrote up that split in running Redis and RabbitMQ together.

Two other comparisons worth having in view. If the workload is an append-only event log that many independent consumers replay at their own pace, Redis Streams and RabbitMQ are both the wrong shape — see RabbitMQ Streams vs Apache Kafka. And if you are weighing an open-source broker against a commercial one for assured delivery, IBM MQ vs RabbitMQ covers that ground.

The failure pattern we see most often is not choosing Redis. It is choosing Redis Pub/Sub for work that needed Streams, or choosing Lists and then discovering three years of accumulated acknowledgement logic nobody wants to touch. Pick the mechanism that matches the guarantee you actually need, and write down which one you picked.

Not sure whether Redis is carrying more queueing responsibility than it should? AceMQ works across both sides of this — Redis support and consulting and RabbitMQ consulting and support. Talk to an AceMQ engineer.

• Redis — Pub/sub, including delivery semantics
• Redis — Streams data type, consumer groups and trimming
• Redis — Persistence: RDB, AOF and fsync policies
• RabbitMQ — Time-to-live and expiration
• RabbitMQ — Dead letter exchanges