Kafka vs Pulsar: Architecture, Operations, and Which to Choose

The Kafka vs Pulsar decision comes down to one architectural choice. Kafka brokers own both serving and storage for the partitions they host. Pulsar separates them — stateless brokers serve traffic while Apache BookKeeper stores data as segmented ledgers. Everything else, from multi-tenancy to scaling behaviour, follows from that split.

Both are mature and both run in production at serious scale. Neither wins the Kafka vs Pulsar comparison outright, and any article that declares one does is usually written by someone selling the other.

Kafka's storage model is a partitioned, replicated log written to broker-local disk. Each partition has a leader broker and follower replicas, and every one of them holds the full partition on its own volume. Serving and storage are the same machine by design, which is what makes Kafka fast and what makes rebalancing expensive.

Pulsar's model is layered. Brokers are described in the documentation as a "stateless component" — they own no data. Storage lives in Apache BookKeeper, "a distributed write-ahead log (WAL) system," where a topic's data is written as a series of ledgers. A ledger is "an append-only data structure with a single writer that is assigned to multiple BookKeeper storage nodes, or bookies," and the "managed ledger" is the storage layer for a single topic, stitching those ledgers together.

Three consequences follow from that split, and they account for most of what teams notice running Kafka vs Pulsar side by side:

• Broker failure is cheap in Pulsar. A broker owns no data, so topic ownership moves to another broker without copying anything. In Kafka a broker loss means a leader election plus, if the broker does not come back with its disk, a full replica rebuild across the network.
• Kafka partitions are bounded by one machine. A partition must fit on the disks of every broker holding a replica. Pulsar's segments are spread across bookies, so a topic is not bounded by any single node's capacity.
• Kafka scales more predictably. One component type, one failure mode to reason about, one set of metrics. Pulsar's independence between compute and storage is real, and so is the extra surface it creates.

Kafka's tiered storage softens the first constraint considerably — more on that below — but the underlying difference in what a broker is does not go away.

This is where most published Kafka vs Pulsar comparisons are out of date, because both projects changed the answer recently. It is also the section worth re-checking against release notes before you commit to anything.

Kafka removed ZooKeeper. Apache Kafka 4.0, released in March 2025, is the first major release to run without it: KRaft mode only, with controllers as Kafka processes rather than a separate ensemble. Pulsar, meanwhile, now supports multiple metadata backends and recommends Oxia, with ZooKeeper still supported and production-ready. So the old line — "Pulsar needs ZooKeeper, Kafka doesn't" — is wrong on both sides now.

What remains true is the component count.

This is Pulsar's clearest structural advantage, and it is not close.

Pulsar was built multi-tenant. A tenant is an administrative unit for allocating capacity and enforcing authentication or authorisation, a tenant contains namespaces, and topics are addressed as persistent://tenant/namespace/topic. Policies that matter operationally — retention, storage quotas — apply at the namespace level, so one team's runaway retention is contained by configuration rather than by a conversation.

Kafka has authentication, ACLs and client quotas, and you can get a long way with a topic naming convention plus disciplined ACLs. But there is no tenancy hierarchy. Isolation is something you enforce; in Pulsar it is something you are given.

If you are building a platform where each customer or each internal team gets its own slice, that difference compounds every quarter. If you are running one pipeline for one team, it is worth nothing at all.

Geo-replication. Pulsar ships with it. Replication is configured per namespace and supports full-mesh, active-active and aggregation topologies. The default is asynchronous — messages are persisted to the local cluster and then replicated by brokers — with synchronous replication available through BookKeeper across data centres at the cost of latency. A separate configuration store quorum coordinates the instance-wide state that makes this work.

Kafka replicates between clusters with MirrorMaker 2, or a vendor equivalent such as Confluent's Cluster Linking. It works, it is widely deployed, and it is also a separate system to deploy, monitor and reason about — including offset translation, which is where most cross-cluster consumer failover surprises live.

Tiered storage. Both have it, and both are production-ready.

• Kafka's implementation (KIP-405) reached production readiness in 3.9, offloading older log segments to pluggable external storage while brokers keep recent data locally.
• Pulsar offloads sealed, read-only segments to Amazon S3, Google Cloud Storage, Azure Blob Storage, Alibaba Cloud OSS or a filesystem. Offloading can be triggered by backlog size, message age, a manual admin command, or a namespace-level policy, and runs asynchronously in the background.

Pulsar's version is more mature simply because it arrived years earlier and its segment-based storage suited it naturally. Kafka's has closed most of the practical gap. For a new deployment in 2026, tiered storage is no longer the Kafka vs Pulsar deciding factor it was in 2022.

Historically this was the most genuine Kafka vs Pulsar capability gap, and it is the one that closed most recently.

Pulsar offers four subscription types: exclusive (the default, one consumer), failover (a primary with standbys), shared (round-robin across consumers, each message to exactly one), and key_shared (messages with the same key always go to the same consumer). It supports individual and cumulative acknowledgement, negative acknowledgement, retry letter topics and dead letter topics, and server-side message deduplication. That set covers work-queue patterns that a classic Kafka consumer group cannot express, because a Kafka partition is assigned to exactly one consumer in the group.

Kafka answered with share groups (KIP-932). They went to early access in 4.0, preview in 4.1, and became production-ready in Apache Kafka 4.2. Consumers in a share group consume cooperatively without pinning a partition to one consumer, with per-record acknowledgement and delivery-attempt counting. That is much closer to Pulsar's shared subscription, and it removes what used to be a real reason to choose Pulsar for queue-shaped workloads.

If your evaluation is based on a comparison written before 2026, check this section against the current release notes — it is the fastest-moving part of the whole comparison.

Kafka, by a wide margin, and this is not a close call worth softening.

Kafka Connect has hundreds of connectors. Kafka Streams and ksqlDB are widely deployed. Every major stream processor — Flink, Spark, and the rest — treats Kafka as a first-class source and sink. Every observability vendor ships Kafka dashboards. Every cloud offers a managed Kafka or a Kafka-compatible endpoint. And, most importantly, you can hire people who have run Kafka in production.

Pulsar has real equivalents: Pulsar IO connectors, Pulsar Functions for lightweight processing, and Pulsar SQL for querying topics. The catalogue is smaller, the third-party integration surface is thinner, and the operational knowledge is concentrated in fewer people and a smaller set of vendors.

For an architecture decision this matters more than most Kafka vs Pulsar write-ups admit. The cost of an unfamiliar platform is not paid at selection time. It is paid at 3 a.m., eighteen months later, by whoever is on call.

Choose Kafka when you want the largest ecosystem and hiring pool, when your workload is a straightforward high-throughput pipeline, when you want the fewest process types to operate, or when you need a managed service on any cloud without a third-party contract.

Choose Pulsar when multi-tenancy is a product requirement rather than a nice-to-have, when you need built-in per-namespace geo-replication across several regions, when storage and compute genuinely need to scale independently, or when a single topic's data would not fit comfortably on one broker's disks.

Do not migrate for elegance. Pulsar's architecture is cleaner in ways that are easy to admire on a whiteboard. If you cannot name the specific constraint it lifts for your workload, the migration will cost you tooling, retraining and institutional knowledge in exchange for a diagram you like better.

The honest summary: this is not a case where one system is behind. The gaps that used to make the Kafka vs Pulsar choice obvious in either direction — ZooKeeper, tiered storage, queue semantics — have all narrowed since 2024. What is left is a genuine architecture question about whether you need serving and storage to scale separately, and a genuine organisational question about which platform your team can actually run.

Evaluating a streaming platform, or supporting one you already run? AceMQ runs vendor-neutral assessments that start from the workload rather than the architecture diagram — see Kafka consulting and support and Apache Kafka support and consulting, or talk to an AceMQ engineer.

• Apache Pulsar — Architecture overview (brokers, BookKeeper, managed ledgers, metadata store)
• Apache Pulsar — Multi-tenancy
• Apache Pulsar — Geo-replication
• Apache Pulsar — Tiered storage
• Apache Kafka — 4.0.0 release announcement (KRaft-only, KIP-848 GA)
• Apache Kafka — 4.2.0 release announcement (share groups production-ready)
• Apache Kafka — Tiered storage