Guide · RabbitMQ

The RabbitMQ Disaster Recovery Guide

High availability keeps a cluster serving when a node dies. Disaster recovery is what happens when the cluster, the site or the region does. This guide takes the two in order, gives the short answer at each step, and points to the article that works it in full.

Tyler Eastridge

By Tyler Eastridge, Head of Operations

LinkedIn · Updated

4 min read7 sections
On this page
DR in one paragraph

DR in one paragraph

Start with the failure you are actually protecting against, because the design differs. A node failure is handled inside the cluster by quorum queues across an odd number of nodes in separate failure domains. A site or region failure needs a second cluster and a replication mechanism, federation or shovel, chosen by whether you need topology mirrored or messages moved. Definitions are backed up separately from messages, because a rebuild starts from definitions. Recovery objectives decide how much of this you build. And none of it counts until the failover has been run, on purpose, with clients connected.

Separate high availability from disaster recovery

HA is redundancy inside one cluster: a node fails and the remaining nodes carry the queues. DR is redundancy across clusters: a site fails and another cluster takes the workload. Quorum queues give you the first; they do nothing for the second, because a cluster does not span regions well and should not be asked to. Decide which failure you are designing for before choosing any mechanism.

Get HA right inside the cluster first

Three or five nodes across independent failure domains, quorum queues for every workload that must survive a node loss, pause_minority partition handling, and classic mirrored queues retired. A DR design built on top of a cluster that still runs mirrored queues inherits their failover behaviour, which is the thing you were trying to fix.

Choose federation or shovel for the second site

Both move messages between clusters over AMQP and both survive link failure; they differ in what they replicate. Federation links exchanges and queues so topology is mirrored and consumers can run on either side; shovel moves messages from a source to a destination with more control and less coupling. For active-passive DR most estates want shovels from specific queues; for active-active or geo-distributed consumers, federated exchanges. Neither gives you exactly-once, so consumers must be idempotent either way.

Back up definitions separately from messages

Users, vhosts, permissions, exchanges, queues, bindings and policies export as one JSON document from the management API, and a new cluster imports it in seconds. Messages do not export that way; they are recovered by the replication in step 3 or accepted as lost within the recovery objective. Export definitions on every change and store them with the infrastructure code, so a rebuild starts from a known topology rather than from memory.

Set recovery objectives before sizing the design

Recovery time objective decides whether the second site runs warm with shovels flowing or is rebuilt from definitions on demand. Recovery point objective decides whether in-flight messages must survive, which pushes you toward publisher confirms, durable queues and replication, or may be lost, which allows a far simpler design. Most RabbitMQ workloads tolerate a short RPO because producers retry; state the number anyway, because it is the number the rest of the design is measured against.

If RabbitMQ runs on Kubernetes, DR is also a storage question

The Cluster Operator handles node replacement inside a cluster, but persistent volumes are zonal or regional depending on the storage class, and a region loss takes the volumes with it. A second cluster in a second region with shovel or federation remains the answer; volume snapshots are a definitions-and-data backup, not a failover. Confirm the storage class's failure domain before trusting the replica count.

Run the failover, on purpose, with clients connected

A documented failover is a hypothesis. Kill a node and confirm quorum queues elect leaders and clients reconnect. Then take the primary site offline and confirm consumers on the secondary receive traffic within the recovery time objective, that definitions imported cleanly, and that the shovels or federation links resume when the primary returns. Record the elapsed time; that is your real RTO. Repeat after every topology change and at least quarterly.

Frequently asked questions

Do quorum queues provide disaster recovery?

No. Quorum queues provide high availability inside one cluster: a node can fail and the queue survives. Disaster recovery needs a second cluster in a separate site or region with federation or shovel replicating to it, plus a definitions backup to rebuild from.

Should I use federation or shovel for RabbitMQ DR?

Shovel for active-passive DR, where messages must move from specific queues to a standby cluster. Federation where consumers run on both sides or topology must be mirrored. Both survive link failures; neither is exactly-once, so consumers need to be idempotent.

Can a RabbitMQ cluster span two regions?

It can, and it should not. Inter-node Erlang distribution assumes low latency, and a WAN link turns ordinary jitter into partitions. Run one cluster per region and replicate between them.

How often should we test RabbitMQ failover?

After every topology or version change, and at least quarterly. The test is a real node kill and a real site failover with clients connected, timed against the recovery objective.

RabbitMQ services

Where this gets done

More resources

Other RabbitMQ guides, comparisons and research

From the blog

Recent RabbitMQ articles

Next step

Need this done on your cluster?

AceMQ's senior RabbitMQ engineers support 130+ enterprise clients in 26+ countries under a 15-minute emergency SLA, with direct escalation to the RabbitMQ core team.