The RabbitMQ on Kubernetes Guide

Kubernetes gives RabbitMQ scheduling, self-healing and a consistent deployment model, and takes away control over placement, networking and restarts unless you claim it back with configuration. It fits estates already operating Kubernetes with persistent workloads. It does not fit a team running Kubernetes for stateless services only, who will meet stateful failure modes for the first time on the message broker.

The RabbitMQ Cluster Operator manages the StatefulSet, the headless service, the Erlang cookie, plugin enablement and rolling updates from a single RabbitmqCluster resource, and the Messaging Topology Operator manages vhosts, users, queues and policies as Kubernetes objects. A hand-written StatefulSet can work, and every one we have inherited has had a rolling-update or peer-discovery defect the Operator already handles. On OpenShift the Operator is the supported path and the security-context constraints need setting explicitly.

Every node needs its own persistent volume, and the storage class decides what a failure looks like: zonal disks pin the pod to a zone and are lost with it, regional disks survive a zone and cost more, and network file systems are too slow for the quorum write-ahead log. Check the failure domain, the IOPS and the reclaim policy before trusting a replica count. Volume snapshots are a backup, not a failover.

Nodes talk over Erlang distribution and discover each other through EPMD, so network policy must allow those ports between the pods as well as AMQP and the management port from clients. Clients should connect through a Service or a load balancer that understands long-lived connections, not one that idles them out. Cluster networks add jitter that a LAN does not; tune heartbeats and net_ticktime to it, or a busy node will be declared dead by its peers.

Requests set to peak memory and CPU, not average, or the scheduler will pack nodes onto hosts that cannot hold them under load. Memory limit above the RabbitMQ watermark with headroom for Erlang, or the kernel kills the node before RabbitMQ can apply flow control. Pod anti-affinity across hosts and topology-spread across zones, so a node failure is a node failure and not a cluster one. PodDisruptionBudget of one, so a cluster upgrade cannot evict two nodes at once.

The Prometheus plugin exposes per-node and per-object metrics; scrape both, and add the Operator's own metrics and the kubelet's for pod restarts and volume health. Five things matter first: queue depth and unacked counts, memory against the watermark, disk free against the limit, connection and channel counts, and partition status. Alert on trends, and correlate a RabbitMQ alarm with pod events, because on Kubernetes the cause is often a reschedule.

The Operator rolls one node at a time and waits for it to rejoin, which is the whole point; the failure mode is an image change plus a node-pool upgrade plus an autoscaler decision landing in the same hour. Sequence platform and broker upgrades, respect the PodDisruptionBudget, and check quorum-queue leader distribution before and after. Version upgrades follow the same hop rules as anywhere else, including the Erlang dependency and the feature-flag sequence.

A broker on Kubernetes needs someone who understands both, and that person is often the one who left. The alternatives to abandoning Kubernetes are a support contract, if the platform team can hold the pager, or a managed service that runs the Operator, the volumes, the network policy and the updates on your cluster while the data stays in your account.