Most Kubernetes estates are healthy for stateless services. Trouble starts when the workload has memory of its own: a message broker with a quorum, a database with a write-ahead log, a cache with a persistence file. Kubernetes treats every pod as replaceable. A broker is not. The four decisions below are where those two views collide, and where most of our calls begin.
Storage decides whether the cluster is stable. A quorum queue or a Kafka log makes a durable write, then waits for the disk to confirm it. If the volume's fsync latency is high or erratic, the broker slows, the quorum leader is lost to a slow follower, and the cluster looks sick when the disk is the cause. Cloud volumes differ in baseline IOPS, burst credits and per-volume ceilings. We benchmark the real PVC with fio before a broker is placed on it, and write the result into the sizing record so the next change to the storage class is a decision with numbers. Binding mode matters too. With WaitForFirstConsumer, a volume is created in the zone where the pod first lands, and a pod rescheduled to a different zone cannot attach it. Anti-affinity and zone spread need to match the volume topology, or a node failure turns into a stuck pod.
Memory limits must account for everything the process uses. A JVM broker has a heap, metaspace, thread stacks, direct memory and native allocations. A limit set from the heap alone gets OOMKilled with no OutOfMemoryError, no dump, and healthy heap graphs. We set heap as a percentage of the limit, cap thread pools and direct buffers, and alert on container working set against the limit.
Disruption budgets are part of the broker design. An upgrade or autoscaler event drains nodes. A PodDisruptionBudget that allows one pod unavailable only protects a three-member quorum if the other two are in sync at that moment. A readiness probe that returns ready as soon as the port opens will let the drain move to the next pod while the last one is still catching up. We write readiness checks that mean "caught up", set budgets per workload, and rehearse a drain in staging with the operations team watching, because the first real drain should not be the first drain.
Upgrades need an order. Platform, operators, CSI and CNI drivers, then workloads. Each has a supported range for the next Kubernetes minor, and stateful workloads go last, one pod at a time, with a rollback at every step. We produce the pre-flight checklist (deprecated APIs in use, webhooks that can block, driver versions), rehearse on a non-production cluster and run the window with your team.
What we do not do on this page: run the cluster for you (see /containers/managed-services/) or host your workloads. Support means that when something breaks at 3am, a senior engineer who knows your storage class, your PDBs and your broker settings joins the bridge.