Is Kafka 4.0 (KRaft, No ZooKeeper) Production-Ready?

Kafka 4.0 represents the most significant architectural shift in the project's history: ZooKeeper, the external coordination service Kafka relied on since its earliest versions, is gone. Cluster metadata and controller consensus are now handled internally via KRaft (Kafka Raft), Kafka's own Raft-based consensus protocol. This is a genuine simplification of the operational model — one fewer distributed system to run and tune — but "newly released major architecture" and "battle-tested for your specific workload" are different claims.

This post covers what's actually changed operationally, based on a real production case that hit a significant performance issue on Kafka 4.0.0 specifically.

KRaft mode is the standard, production architecture in Kafka 4.0 — it's not a beta or preview feature at this point. Organizations are actively deploying it in production, including for demanding use cases: one organization specifically adopted Kafka 4.0.0 for a banking-sector transaction processing platform explicitly because it removes the ZooKeeper dependency, treating that architectural simplification as a deliberate, forward-looking technology choice rather than a risk to avoid.

That said, "the architecture is production-grade" and "every feature behaves identically to how it did under the ZooKeeper-based model" are different claims — and the real case referenced throughout this post is a direct illustration of exactly that gap.

A significant, specific performance regression tied to Kafka transactions. An organization running a transaction-processing platform on Kafka 4.0.0, using Spring Boot 3.2 and Java 17, had previously measured throughput around 320,000 transactions per hour using standard Kafka messaging. After adding recovery capability to their system — implemented using Kafka's transactional messaging feature — throughput dropped to 40,000-50,000 transactions per hour, and critically, scaling up the number of service instances did not improve it. The system was stuck at the same throughput ceiling regardless of horizontal scaling, which is a strong signal that the bottleneck was structural (likely coordinator-related) rather than simple resource contention.

When the team tested running the same workload without Kafka transactions, performance returned to normal — isolating the regression specifically to the transactional messaging path, not to KRaft mode, the broker configuration, or the application code more broadly.

Not necessarily broken — but it does mean transactions specifically warrant deeper validation before you commit to them for a performance-sensitive workload, since the transaction coordinator's behavior under KRaft is architecturally different from how it worked under ZooKeeper-based clusters, and that difference can manifest as exactly this kind of throughput ceiling under certain usage patterns.

What we know from this case:

• The throughput ceiling was reproducible and consistent (not intermittent)
• It didn't respond to horizontal scaling, ruling out simple resource starvation as the cause
• Removing transactions from the same workload restored normal throughput

What this points to diagnostically: transaction coordinator behavior — how transactional state is tracked, committed, and coordinated across the cluster — is a strong candidate for the bottleneck, since that's the component most directly tied to the "transactions on vs. off" behavior difference observed. A structured diagnostic engagement for this kind of issue would examine transaction coordinator placement, transaction timeout configuration, and transactional.id cardinality and partitioning as the first investigation targets.

Not necessarily — but validate your specific transactional usage pattern under realistic load before committing to it for a production-critical, latency-sensitive workload, rather than assuming general Kafka transaction guidance from pre-4.0 documentation carries over unchanged.

If your use case requires Kafka transactions (exactly-once semantics across produce/consume boundaries, or atomic multi-partition writes):

• Load test the transactional path specifically and separately from your non-transactional throughput baseline, using a representative transaction rate and transactional.id pattern
• Test whether horizontal scaling of your producer/consumer instances actually improves transactional throughput in your environment — if it doesn't, that's the same warning signal observed in the case above, worth investigating before going to production
• Review current Kafka 4.x release notes and known issues specifically for KRaft transaction coordinator behavior, since this is an actively evolving area of the codebase post-ZooKeeper-removal

If your use case doesn't strictly require transactions, evaluate whether your recovery/idempotency requirements can be met through alternative patterns — idempotent producer configuration alone (without full transactions), application-level deduplication, or a different recovery architecture — that may avoid the specific bottleneck observed here entirely.

The real case referenced throughout this post used JMeter to generate controlled, steady load at a specific transaction-per-second/minute rate — a deliberate choice specifically because it gives clear, controllable visibility into how the system responds to load ramps, rather than relying on production traffic patterns that are harder to isolate and reproduce for diagnostic purposes.

A sound validation approach:

1. Establish a non-transactional throughput baseline first, so you have a clean comparison point
2. Introduce transactions and measure the throughput delta under the same load profile
3. Test horizontal scaling explicitly — add producer/consumer instances and confirm throughput actually increases; if it plateaus, that's your signal to investigate coordinator-level bottlenecks before going further
4. Use a load-generation tool that lets you hold a steady, known transaction rate (JMeter, or Kafka's own kafka-producer-perf-test.sh for producer-side validation) rather than relying solely on organic traffic during a soft launch

If you're currently on a pre-KRaft version and planning your upgrade path, Kafka 4.0 and later are ZooKeeper-free by design — ZooKeeper support was removed, not just deprecated, as of Kafka 4.0. This means your upgrade planning needs to include a KRaft migration as part of the path, not as an optional follow-on step.

For workloads that don't use Kafka transactions heavily, this migration is generally lower-risk than the case described in this post, since the specific bottleneck identified here was isolated to transactional throughput specifically, not general KRaft cluster operation. For transaction-heavy workloads, budget explicit load-testing time for the transactional path as part of your migration plan, using the validation approach outlined above, rather than assuming pre-KRaft transaction performance characteristics carry over unchanged.

The core KRaft architecture is production-ready and is being actively adopted for demanding workloads, including financial transaction processing. Kafka transactions specifically warrant dedicated load validation before you commit to them for a performance-sensitive production workload, based directly on a real, reproducible throughput regression observed in production testing. This isn't a reason to avoid Kafka 4.0 — it's a reason to test your specific usage pattern deliberately rather than assuming full behavioral parity with pre-KRaft Kafka versions across every feature, transactions included.

Migrating to Kafka 4.0, or seeing an unexplained throughput ceiling with Kafka transactions in your KRaft-based cluster? Contact AceMQ for a diagnostic engagement. Not sure where to start? Talk to an AceMQ engineer.

• Kafka Support
• What Actually Causes a Kafka Producer Timeout
• Kafka Consumer Lag: Causes, Monitoring, and How to Reduce It
• AceMQ Home