The RabbitMQ Performance Tuning Guide

Most RabbitMQ tuning fails because the change was made before the bottleneck was found. Get three things on a graph first: publish and deliver rates per queue, publisher confirm latency measured in the application, and node-level CPU, memory, disk latency and network throughput. RabbitMQ's own production guidelines put it bluntly: monitoring, benchmarking and measuring are required to find the best setting for an environment. Then reproduce the load with PerfTest, the RabbitMQ team's load-testing tool, so that each change is tested against the same traffic.

The documented production minimum per node is 4 CPU cores and 4 GiB of RAM, with no other data service or I/O-heavy application on the host. RabbitMQ was not designed for single-core environments. A surprising share of slow clusters turn out to be under this floor or sharing a host with a database.

Storage matters more than it used to, because quorum queues and streams are built for durable storage with stable latency. Prefer local SSD or NVMe to network-attached storage, never share a data directory between nodes, and do not put node data on a distributed filesystem: the storage layer assumes local fsync semantics that those filesystems often do not honour.

By default a node stops accepting new messages when it uses more than 60% of available memory (vm_memory_high_watermark.relative = 0.6). The recommended range is 0.4 to 0.7. Going above 0.7 is the classic false economy: the operating system and filesystem need at least 30% of memory for the page cache, and starving them causes paging that slows every disk operation the broker makes. If publishers are being blocked, the fix is almost always faster consumers or more nodes, not a higher watermark.

The default disk_free_limit is 50 MB, a value chosen so that RabbitMQ boots on small development partitions. In production it is far too low: a node that runs out of disk fails writes and can lose data. The guideline is to set the limit to roughly the size of the memory watermark, so a node with a 4 GB watermark gets disk_free_limit.absolute = 4G. Overprovision the disk as well, because quorum queues and streams do not always reclaim space from consumed messages quickly.

Every connection and every queue uses file descriptors. Allow at least 50,000 for the RabbitMQ user. The sizing rule from the documentation: take the 95th percentile of concurrent connections, multiply by two, and add the number of queues. Limits as high as 500,000 cost almost nothing and are recommended for production. A node that hits this limit stops accepting connections, which looks like a performance problem and is not one.

Application behaviour is the largest single source of RabbitMQ performance problems, and no broker setting compensates for it. The rules are few. Connections and channels should be long-lived: opening a connection per message is expensive and causes connection churn that exhausts file handles and memory. Use separate connections for publishing and consuming, so consumers are not caught in flow control applied to a publisher. Do not set heartbeat timeouts below 5 seconds, which produces false disconnects under load. Use the client library's automatic connection recovery where it exists. And avoid polling with basic.get; it is inherently inefficient compared with a subscribed consumer.

From our engagements. In our ticket log, 33 of 145 RabbitMQ tickets (23%) were message flow, load balancer, connection or client-setting problems, against 8 (6%) for memory, disk, CPU or file descriptor limits (the ticket analysis). Where a team reports that the broker is slow, the cause is most often a prefetch that is misconfigured, acknowledgements that are synchronous, or one slow consumer holding the head of a queue. On a telecom IoT platform with 300,000+ connected devices that crashed weekly out of memory, the fixes AceMQ made included raising prefetch from 1 to 20, adding per-queue memory limits and TTLs, and recommending scale-out rather than scale-up (the case).

Prefetch is the consumer-side setting with the most effect on throughput. Too low and consumers sit idle waiting for the network; too high and one slow consumer hoards messages while others starve, and memory climbs with the unacknowledged backlog. Start from the processing time per message, measure consumer utilisation, and adjust. Acknowledge in batches where ordering allows it.

Queue type sets the performance ceiling. Quorum queues replicate and flush every write, so their throughput falls as replica count rises; replicate to a majority of nodes rather than all of them on larger clusters. Streams were designed for throughput and large backlogs and use little broker memory, at the cost of queue features such as dead-lettering. Classic queues are for transient work only. A queue that must hold millions of messages is a stream, not a bigger quorum queue.

The documentation gives a rule of thumb for the minimum bandwidth each node needs: 95th percentile message rate, times 95th percentile message size, times 110% for protocol overhead, times 8 bits. At 20,000 messages a second and 6 KB payloads that is just over 1 gigabit per second, before replication traffic between nodes. TLS adds measurable CPU cost on both broker and clients, so include it in the benchmark rather than adding it afterwards.

Use an odd number of nodes so a majority is always identifiable. More nodes help with connection counts, provided a load balancer spreads the connections. They do not help indefinitely: all metadata is replicated to every node and most metadata changes are synchronous, so the cost of each declaration grows with cluster size. The documented advice for anyone heading toward double-digit node counts is to split the system into independent clusters instead.