RabbitMQ cluster sizing calculator for quorum queues and streams

The calculator sizes one node and shows how it got there; every node in the cluster holds a full copy of the data, so the per-node figure applies to all of them. Use the 95th percentile for rate and message size, and set the hours to the consumer outage you want the cluster to ride out (quorum queues) or the retention window (streams).

It sizes the data path only. For node placement, partition behaviour and policies, see the clustering and sizing guide; for node counts with failure domains worked through, see cluster sizing for HA and DR.

For 1,000 messages a second of 4 KiB held for 4 hours on a three-node cluster with quorum queues, each node needs 6 GiB of RAM, 114 GiB of disk and 4 cores. The table is what the calculator shows on first load; change an input and it recomputes the same steps.

Nine of the twelve rules are straight from the RabbitMQ documentation; three are AceMQ planning practice, marked as such, and you should change them if your own measurements disagree. The calculator does not hide a constant: each one below is the one in the arithmetic.

Use three nodes to survive one failure and five to survive two, and do not go past five for quorum queues without a reason. A quorum queue needs more than half its members up, so the tolerated failures follow the table below. The docs note that quorum queue performance tails off quite a bit above five nodes and that they do not recommend quorum queues on more than seven nodes, and that clusters of double-digit node counts should usually become separate clusters.

Streams are quorum systems too, with a replica on each node by default, so the same odd-number rule holds. On the current series the metadata store and quorum queues are both Raft-based, so the old pause_minority advice from the mirrored-queue era no longer applies; the failure patterns guide covers partitions on 4.x.

The calculator sizes the two replicated types, quorum queues and streams, because those are the ones that need a cluster to be safe. Classic mirrored queues no longer exist: classic queue mirroring was removed in RabbitMQ 4.0, so a cluster that relied on mirroring is sized as quorum queues or streams.

Use cases that need high availability use a replicated queue, meaning a quorum queue or a stream. You configure the type per queue with the x-queue-type argument at declaration, or set default_queue_type for a node or virtual host, and it cannot be changed afterwards. Exchanges only route messages and hold none, so they add nothing to the data-path figures; the queues and streams they feed do, and the RabbitMQ stream type suits the use cases where many consumers need to read the same long retention window.

A quorum queue is designed for data safety: a confirmed message should not be lost as long as a majority of the nodes hosting the queue stay available. A few behaviours around restarts and consumers decide whether the sized cluster rides out a bad hour.

• Broker restart. A follower that fails or restarts resumes log replication where it left off, with no special catch-up process. During a broker restart on a three-node cluster the queue has no failure tolerance left, so restart one node at a time and in a quiet period.
• Acknowledgements. Use publisher confirms, and have consumers consume with manual acknowledgements. Messages not yet confirmed to the publisher have no safety guarantee.
• Prefetch. Quorum queues need a per-consumer prefetch, because global QoS prefetch is not supported. Too low a value starves consumers; values in the thousands can overload them. This is the setting that decides how much a consumer holds unacknowledged when it reconnects.
• Redelivery. Unacknowledged messages return to the queue when a consumer drops. Since 4.0 a quorum queue's delivery limit defaults to 20, after which a message is dropped or dead-lettered, so a poison message does not cycle forever.
• Several queues, not one. A single queue handles a bounded rate, so one busy queue will not use all the cores you sized. Spread a high publishing rate over multiple queues, and review the topology if you expect more than about 5,000 quorum queues.
• Features that carry over. Single active consumer, dead lettering, and from 4.3 strict priorities with 32 levels all work on quorum queues. Both AMQP 0-9-1 and AMQP 1.0 clients can use them.

The calculator sizes one cluster in one site. If the data must live in two regions, do not stretch a replicated queue across them: size a cluster per region and move messages between them with federation or a shovel, which tolerate a slow or intermittent link. The operating rules for those links are in the federation and shovel runbook.

Check the sized figures against what the cluster does under load. The management UI and HTTP API show queue length (messages ready for delivery), ingress and egress rates, consumer counts and ready against unacknowledged messages per queue, and rabbitmqctl list_queues gives the basic figures from the command line. Compare the busiest-hour queue length with the backlog you sized for, and treat a memory or disk alarm as a sign the model is wrong for your workload, not as a threshold to raise.

It does not size connections, queue counts, bursts, TLS or classic queues, and each of those can move the answer more than the averages do. Add what you can measure:

• Classic queues. They are not replicated, so a node loss takes their data with it; use quorum queues or streams for anything you cannot lose. The classic queue guide describes a bounded in-memory window, but its real cost depends on queue count, which the calculator does not ask for.
• Connections and file handles. The production checklist's rule is 2 times the 95th percentile of concurrent connections plus the number of queues, with at least 50,000 file descriptors for the RabbitMQ user. Measure the memory these use and enter it as other memory.
• Bursts. The calculator takes a sustained rate. A burst is absorbed by the backlog figure only if consumers catch up afterwards, so check how long they take.
• TLS and CPU. The production checklist notes TLS can have a significant impact on CPU use. The core count here is a floor, not a forecast.
• Containers. The memory guide recommends an absolute watermark under Kubernetes, because a node cannot always detect the effective cgroup limit. Set vm_memory_high_watermark.absolute from the RAM figure above.

In our reviews we size to the backlog a consumer outage produces, not to steady state, because steady state is not what fills a disk.

Treat the output as the number you bring to a design review, and have it checked against your queue count, message shapes and consumer behaviour before you buy hardware. Send your rate, message profile and retention requirement to an AceMQ engineer, who will tell you where the model is wrong for your workload, including when you need less than it suggests. Enquiries to AceMQ get a reply within 1 hour through RabbitMQ implementation, and the RabbitMQ health check reviews a running cluster against these same rules.