Kafka on Kubernetes: EKS vs AKS vs GKE

Kafka on Kubernetes works, but it is a stateful workload wearing a stateless platform's clothes. Brokers carry identity, data and partition leadership, so pod churn is never free. EKS, AKS and GKE differ mainly in storage class semantics, zone-redundant disk options and load balancer behavior — the Kafka-side design is identical on all three.

The provider choice is the smaller half of the decision. Most of what goes wrong with Kafka on Kubernetes goes wrong the same way on every cloud: a broker comes back with a different identity, a replica set collapses into one zone, or a client bootstraps successfully and then cannot reach a single broker.

If you are weighing the same question for RabbitMQ, we covered running RabbitMQ on EKS, AKS and GKE separately — the storage reasoning rhymes, the failure modes do not.

Three properties separate Kafka on Kubernetes from the deployments Kubernetes was designed around.

Brokers have identity. A broker is not interchangeable with its replacement. It owns a node ID, a set of partition replicas, and leadership for some of them. Kubernetes wants to treat a pod as fungible; Kafka does not. That is why brokers run as StatefulSets with stable network identities and per-pod volumes rather than as a Deployment.

Brokers own data. Every broker's log directory holds partition segments that are expensive to rebuild. A pod that returns without its old volume is not a restart, it is a full replica rebuild — pulled from the leader across the network, competing with live traffic for the same bandwidth. On a large broker that is hours, not seconds.

Pod churn triggers Kafka work. Node autoscaling, spot reclamation, a rolling upgrade, an OOM kill: each takes a broker out, moves leadership, and shrinks the in-sync replica set while it is gone. Consumer groups feel it too — what triggers a Kafka rebalance covers the consumer side of the same event.

So a Kubernetes cluster tuned for fast, aggressive pod movement is actively hostile to Kafka. You want the opposite: PodDisruptionBudgets that refuse to take a second broker down while the first is still catching up, and rolling updates that wait for ISR to recover rather than for a readiness probe to go green.

One thing has got easier. Apache Kafka 4.0 removed ZooKeeper mode entirely and runs KRaft only, so there is no separate ensemble alongside the brokers. That is one less StatefulSet — though controllers are still stateful processes needing their own volumes and disruption budget.

Strimzi is the main open-source Kafka Kubernetes operator and the one most estates end up on. Check the current feature set against the docs for the version you are installing — custom resource shapes have changed across API versions — but as of writing it provides:

• Cluster Operator for brokers and controllers, plus Kafka Connect, MirrorMaker 2 and the HTTP Bridge.
• Topic Operator and User Operator, so topics and ACLs are declared as KafkaTopic and KafkaUser resources instead of created by a script nobody can find later.
• Node pools (KafkaNodePool), giving controllers and brokers different roles, storage and instance shapes inside one logical cluster.
• Drain Cleaner, which intercepts pod evictions so the operator moves brokers deliberately rather than letting the node drain do it.
• Cruise Control integration for partition rebalancing, and a tieredStorage property for offloading older segments to object storage.

The commercial distributions — Confluent Platform's operator, Red Hat's build of Strimzi — wrap the same ideas with a support contract. Choosing between them is a procurement question.

What an operator buys you is the boring, error-prone parts: ordered rolling restarts that respect ISR, certificate rotation, and generating advertised.listeners correctly for each listener type. Running Kafka on Kubernetes with raw StatefulSets is possible and we have seen it done. It is also where most of the incidents come from.

This is where the providers actually diverge. The Kafka configuration does not change; the volume underneath it does.

Kafka has had a rack concept since long before anyone ran it on Kubernetes, and it maps onto cloud availability zones almost perfectly.

Set broker.rack on each broker to the zone it is running in. Kafka's replica assignment then spreads replicas so that, in the Apache documentation's words, a partition will span min(#racks, replication-factor) different racks. With three zones and replication factor 3, every partition has a replica in each zone, and losing a zone costs you one replica per partition rather than all of them.

For Kafka on Kubernetes you do not set this by hand. Strimzi reads a node label — conventionally topology.kubernetes.io/zone — and injects the value as broker.rack for each broker pod. That requires the operator to read cluster-scoped Node objects, so RBAC has to allow it; a rack configuration that silently does nothing is usually a missing ClusterRoleBinding.

Two traps worth naming:

Uneven brokers per zone skews replica placement. Apache's own guidance is blunt: racks with fewer brokers get more replicas, so they use more storage and spend more on replication. Six brokers as 3/2/1 across zones is worse than 2/2/2, even though the totals match.

Rack awareness is not zone anti-affinity. broker.rack tells Kafka how to place replicas. It does not tell Kubernetes how to place pods. You need both: topologySpreadConstraints (or pod anti-affinity) so the scheduler distributes brokers across zones, and broker.rack so Kafka knows what that distribution means.

Once racks are set, the same labels let consumers fetch from the closest replica. Set client.rack on consumers and replica.selector.class to org.apache.kafka.common.replica.RackAwareReplicaSelector on the brokers, and a consumer reads from an in-zone follower rather than crossing a zone to reach the leader. On a chatty cluster that is a real cross-zone transfer line item, removed by one configuration change.

This is the classic failure. It looks like a network problem, it is a metadata problem, and it behaves identically on all three clouds.

Kafka clients do not keep talking to whatever address you gave them. They connect to a bootstrap server, ask for cluster metadata, and receive a list of broker addresses — the values in each broker's advertised.listeners. From that point on they connect directly to those addresses. The bootstrap connection succeeded; the ones that matter never do.

Inside Kubernetes the natural addresses are pod IPs and headless-service DNS names. Both are meaningless outside the cluster. So the symptom is a client that authenticates, fetches metadata, then times out against hosts it cannot resolve — while kubectl exec into any broker shows a perfectly healthy cluster.

The fix is to give each listener an advertised address that is correct for the clients using that listener, which means separate listeners:

• Internal listener — headless service DNS, for in-cluster producers, consumers and inter-broker traffic.
• External listener — per-broker addressable endpoints. Not one load balancer in front of all brokers: clients must reach a specific broker, because partition leadership is per-broker. Operators implement this as a load balancer or node port per broker, plus a bootstrap endpoint.

advertised.listeners defaults to null and listeners defaults to PLAINTEXT://:9092, so anything here is a deliberate override. Let the operator generate it. Hand-rolling advertised addresses is how a Kafka on Kubernetes cluster works right up until the day it scales.

Two related points: put the external listener behind TLS and real authentication before it exists at all, and remember a per-broker load balancer set is a per-broker bill on every cloud. That cost is a common reason teams end up back at the managed-service conversation.

Honest answer: more often than engineers like to admit, and less often than vendors claim.

Managed wins when Kafka would be the first stateful workload on your cluster, when nobody on the team has taken a broker through a disk-full incident, or when the engineering time you would spend on upgrades and rebalances is worth more elsewhere. The first-party options differ by cloud: Amazon MSK offers Standard and Express brokers (Express replicates across three AZs and is only available in a three-AZ configuration) plus MSK Serverless; Google Cloud Managed Service for Apache Kafka is generally available; Azure has no first-party Apache Kafka service, so the comparison there is Event Hubs' Kafka protocol endpoint — which speaks Kafka 1.0 and later but is not a Kafka cluster and does not behave like one in every corner — or a third party such as Confluent Cloud.

Self-managed wins when you already run stateful workloads on Kubernetes competently, when you need broker-level configuration a managed tier will not expose, when data residency or network topology rules it out, or when your volume makes the managed premium genuinely large.

The decision rarely turns on the cloud provider. It turns on whether your team's Kubernetes maturity is real. On a recent payments engagement we ran a RabbitMQ upgrade using Kubernetes sidecar deployments and a staged migration to reach zero downtime — and that customer was evaluating managed options in parallel, which is the right instinct. Both questions deserve an answer before you commit.

Four signals separate a healthy Kafka on Kubernetes cluster from one quietly degrading:

• Under-replicated partitions. The single most useful number. A value that does not return to zero after a rolling restart means a broker is not keeping up.
• Volume utilization per broker, not per cluster. Averages hide the one broker about to fill, and a full log directory takes a broker offline.
• Pod restart counts and eviction events. Correlate against under-replication. A node pool recycling nodes on a schedule you did not choose shows up here first.
• Consumer lag. The user-visible symptom of all of the above — diagnosing Kafka consumer lag covers reading it properly.

Tiered storage (KIP-405, production-ready as of Kafka 3.9) changes disk sizing considerably if retention is long: older segments move to object storage, brokers keep less locally, and replica rebuilds get shorter. Worth evaluating before buying another terabyte per broker.

The Kafka on EKS clusters we build for enterprise clients share one shape, because the failure modes on AWS are predictable and the fixes are cheap when designed in from the start.

Dedicated broker node groups. Brokers get their own managed node group, tainted so nothing else schedules there, with a matching toleration on the Kafka pods. Shared nodes let a noisy batch job steal page cache and network from a broker, and page cache is most of what makes Kafka fast. One node group per availability zone lets you upgrade zone by zone.

gp3, not gp2. gp2 ties IOPS to volume size, so a 500 GB log directory gets 1,500 IOPS whether it needs them or not, and burst credits run out under sustained load. gp3 gives a baseline of 3,000 IOPS and 125 MB/s independent of size, and both can be raised without resizing the volume. That is the right model for a broker: you size storage for retention and provision throughput for traffic, separately. We reserve io2 for the hottest clusters, where a broker must sustain hundreds of MB/s with low tail latency.

EBS CSI driver and a zone-aware StorageClass. The in-tree EBS provisioner is gone, so the EBS CSI add-on is required. The StorageClass must set volumeBindingMode: WaitForFirstConsumer. Without it the PV is created before the pod is scheduled, in whatever zone the provisioner picks, and the pod then cannot start because EBS volumes cannot cross zones.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: kafka-gp3
provisioner: ebs.csi.aws.com
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
parameters:
  type: gp3
  iops: "6000"
  throughput: "250"

One broker per zone, enforced twice. A topology spread constraint on topology.kubernetes.io/zone with maxSkew: 1 and whenUnsatisfiable: DoNotSchedule keeps the pods spread. Rack awareness with the same key tells Kafka to place replicas across those zones. You need both: the scheduler does not know about partitions and Kafka does not know about nodes.

IAM roles for service accounts. Anything that talks to AWS, whether a Connect worker writing to S3, MirrorMaker 2, or the CSI driver itself, gets an IRSA role scoped to what it needs. No broad instance profiles, no static keys in secrets.

Listeners for clients outside the cluster. For clients in the same VPC we prefer an internal listener exposed through an internal NLB with one target group per broker, so the bootstrap address and each broker's advertised address resolve to the right pod. Cross-account and on-premises clients reach the same NLB over a private link. We never put a public load balancer in front of brokers.

KRaft, Strimzi, and the Kafka resource. New clusters are KRaft only; see our note on Kafka 4 and KRaft production readiness. With Strimzi node pools the broker definition comes down to this:

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
  name: broker
  labels:
    strimzi.io/cluster: prod
spec:
  replicas: 3
  roles: [broker]
  storage:
    type: persistent-claim
    size: 1Ti
    class: kafka-gp3
    deleteClaim: false
  template:
    pod:
      tolerations:
        - key: workload
          value: kafka
          effect: NoSchedule
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              strimzi.io/cluster: prod
---
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: prod
  annotations:
    strimzi.io/kraft: enabled
    strimzi.io/node-pools: enabled
spec:
  kafka:
    version: 4.0.0
    rack:
      topologyKey: topology.kubernetes.io/zone
    config:
      default.replication.factor: 3
      min.insync.replicas: 2

Partition counts and replication factor are decided per topic, not per cluster; our Kafka partition strategy guide covers that side.

What MSK gives you. AWS patches the brokers, handles storage, and replaces failed instances. IAM authentication and CloudWatch metrics work out of the box, and security teams already know how to audit them. You pay per broker hour plus storage, and the bill is predictable. What you give up is control: a curated set of Kafka versions that lag upstream, a limited set of broker configuration properties, no custom plugins on the broker, and a service that only exists on AWS.

What self-managed on EKS gives you. Any Kafka version the day it ships, any broker configuration, any authorizer or metrics reporter, and the same manifests portable to another cloud or a data centre. It costs engineering time: someone has to own upgrades, storage, monitoring, and the 3 a.m. page. If that someone does not exist, the cluster decays.

Decision rules we use.

• Team shape. No engineer who has run stateful workloads on Kubernetes before: MSK. A platform team that already owns EKS and operators: EKS is a small increment.
• Compliance. Data residency in a region MSK covers and audit requirements met by IAM and CloudTrail: MSK is easier to sign off. Need for custom authorizers, specific TLS cipher control, or portability clauses in a regulator's expectations: EKS.
• Throughput. Under a few hundred MB/s on either. Above that, MSK broker sizing gets expensive fast and tuning options run out; self-managed lets you pick instance types and storage directly.
• Product surface or plumbing. If Kafka is plumbing between internal services, buy it. If Kafka is part of what you sell, whether an event API, a streaming platform, or a multi-tenant data product, you will want version control, plugin control, and the option to leave, so run it.

The two EKS failure modes we get called about. Both are self-inflicted and both are avoidable with the setup above.

The first is a zone outage that takes the only broker in that zone, followed by the discovery that its PV cannot reattach anywhere else. The EBS volume is pinned to the dead zone, so the rescheduled pod sits Pending. With replication factor 3 and min.insync.replicas 2 the cluster keeps serving, but recovery, a fresh volume in a healthy zone and a rebuild from replicas, has to be rehearsed. Clusters with brokers packed into two zones, or with a topic at replication factor 2, lose data here. Our Kafka disaster recovery post covers the rehearsal.

The second is a managed node group upgrade that drains nodes on its own schedule. The default drain will evict one broker, wait for the pod disruption budget, and move on as soon as the replacement pod is Running, which is not the same as caught up. Two evictions in quick succession leave partitions with a single in-sync replica and producers with acks=all start failing. The fix is Strimzi Drain Cleaner, which blocks eviction until the broker has no under-replicated partitions, and a node group maxUnavailable of 1.

If you want a second pair of eyes on an EKS design, our Kubernetes consulting and Kafka consulting teams do this together, and Kafka support covers the cluster afterwards with 24/7 coverage and a 15-minute P1 response.

Planning a Kafka deployment on EKS, AKS or GKE — or trying to work out whether you should be self-managing at all? AceMQ runs Kafka assessments that cover exactly this decision, including the storage, rack and listener design that most tutorials skip. See Apache Kafka support and consulting and our container and Kubernetes messaging services, or talk to an AceMQ engineer.

• Apache Kafka — Broker configuration reference
• Apache Kafka — Basic operations, balancing replicas across racks
• Apache Kafka — 4.0.0 release announcement (KRaft-only, ZooKeeper removed)
• Strimzi — Configuring Strimzi (storage, rack, node pools, tiered storage)
• AWS — Amazon MSK broker types
• Microsoft — Storage concepts in Azure Kubernetes Service
• Google Cloud — GKE persistent volumes and StorageClasses