Kafka vs Pulsar: Architecture, Operations, and Which to Choose

The Kafka vs Pulsar decision comes down to one architectural choice. Kafka brokers own both serving and storage for the partitions they host. Pulsar separates them — stateless brokers serve traffic while Apache BookKeeper stores data as segmented ledgers. Everything else, from multi-tenancy to scaling behavior, follows from that split.

Both are mature and both run in production at serious scale. Neither wins the Kafka vs Pulsar comparison outright, and any article that declares one does is usually written by someone selling the other.

Both platforms are demanding to operate. AceMQ provides 24/7 Kafka support and Apache Pulsar support, including BookKeeper and ZooKeeper, with a 15-minute emergency SLA.

Kafka's storage model is a partitioned, replicated log written to broker-local disk. Each partition has a leader broker and follower replicas, and every one of them holds the full partition on its own volume. Serving and storage are the same machine by design, which is what makes Kafka fast and what makes rebalancing expensive.

Pulsar's model is layered. Brokers are described in the documentation as a "stateless component" — they own no data. Storage lives in Apache BookKeeper, "a distributed write-ahead log (WAL) system," where a topic's data is written as a series of ledgers. A ledger is "an append-only data structure with a single writer that is assigned to multiple BookKeeper storage nodes, or bookies," and the "managed ledger" is the storage layer for a single topic, stitching those ledgers together.

Three consequences follow from that split, and they account for most of what teams notice running Kafka vs Pulsar side by side:

• Broker failure is cheap in Pulsar. A broker owns no data, so topic ownership moves to another broker without copying anything. In Kafka a broker loss means a leader election plus, if the broker does not come back with its disk, a full replica rebuild across the network.
• Kafka partitions are bounded by one machine. A partition must fit on the disks of every broker holding a replica. Pulsar's segments are spread across bookies, so a topic is not bounded by any single node's capacity.
• Kafka scales more predictably. One component type, one failure mode to reason about, one set of metrics. Pulsar's independence between compute and storage is real, and so is the extra surface it creates.

Kafka's tiered storage softens the first constraint considerably — more on that below — but the underlying difference in what a broker is does not go away.

This is where most published Kafka vs Pulsar comparisons are out of date, because both projects changed the answer recently. It is also the section worth re-checking against release notes before you commit to anything.

Kafka removed ZooKeeper. Apache Kafka 4.0, released in March 2025, is the first major release to run without it: KRaft mode only, with controllers as Kafka processes rather than a separate ensemble. Pulsar, meanwhile, now supports multiple metadata backends and recommends Oxia, with ZooKeeper still supported and production-ready. So the old line — "Pulsar needs ZooKeeper, Kafka doesn't" — is wrong on both sides now.

What remains true is the component count.

This is Pulsar's clearest structural advantage, and it is not close.

Pulsar was built multi-tenant. A tenant is an administrative unit for allocating capacity and enforcing authentication or authorization, a tenant contains namespaces, and topics are addressed as persistent://tenant/namespace/topic. Policies that matter operationally — retention, storage quotas — apply at the namespace level, so one team's runaway retention is contained by configuration rather than by a conversation.

Kafka has authentication, ACLs and client quotas, and you can get a long way with a topic naming convention plus disciplined ACLs. But there is no tenancy hierarchy. Isolation is something you enforce; in Pulsar it is something you are given.

If you are building a platform where each customer or each internal team gets its own slice, that difference compounds every quarter. If you are running one pipeline for one team, it is worth nothing at all.

Geo-replication. Pulsar ships with it. Replication is configured per namespace and supports full-mesh, active-active and aggregation topologies. The default is asynchronous — messages are persisted to the local cluster and then replicated by brokers — with synchronous replication available through BookKeeper across data centers at the cost of latency. A separate configuration store quorum coordinates the instance-wide state that makes this work.

Kafka replicates between clusters with MirrorMaker 2, or a vendor equivalent such as Confluent's Cluster Linking. It works, it is widely deployed, and it is also a separate system to deploy, monitor and reason about — including offset translation, which is where most cross-cluster consumer failover surprises live.

Tiered storage. Both have it, and both are production-ready.

• Kafka's implementation (KIP-405) reached production readiness in 3.9, offloading older log segments to pluggable external storage while brokers keep recent data locally.
• Pulsar offloads sealed, read-only segments to Amazon S3, Google Cloud Storage, Azure Blob Storage, Alibaba Cloud OSS or a filesystem. Offloading can be triggered by backlog size, message age, a manual admin command, or a namespace-level policy, and runs asynchronously in the background.

Pulsar's version is more mature simply because it arrived years earlier and its segment-based storage suited it naturally. Kafka's has closed most of the practical gap. For a new deployment in 2026, tiered storage is no longer the Kafka vs Pulsar deciding factor it was in 2022.

Historically this was the most genuine Kafka vs Pulsar capability gap, and it is the one that closed most recently.

Pulsar offers four subscription types: exclusive (the default, one consumer), failover (a primary with standbys), shared (round-robin across consumers, each message to exactly one), and key_shared (messages with the same key always go to the same consumer). It supports individual and cumulative acknowledgement, negative acknowledgement, retry letter topics and dead letter topics, and server-side message deduplication. That set covers work-queue patterns that a classic Kafka consumer group cannot express, because a Kafka partition is assigned to exactly one consumer in the group.

Kafka answered with share groups (KIP-932). They went to early access in 4.0, preview in 4.1, and became production-ready in Apache Kafka 4.2. Consumers in a share group consume cooperatively without pinning a partition to one consumer, with per-record acknowledgement and delivery-attempt counting. That is much closer to Pulsar's shared subscription, and it removes what used to be a real reason to choose Pulsar for queue-shaped workloads.

If your evaluation is based on a comparison written before 2026, check this section against the current release notes — it is the fastest-moving part of the whole comparison.

Kafka, by a wide margin, and this is not a close call worth softening.

Kafka Connect has hundreds of connectors. Kafka Streams and ksqlDB are widely deployed. Every major stream processor — Flink, Spark, and the rest — treats Kafka as a first-class source and sink. Every observability vendor ships Kafka dashboards. Every cloud offers a managed Kafka or a Kafka-compatible endpoint. And, most importantly, you can hire people who have run Kafka in production.

Pulsar has real equivalents: Pulsar IO connectors, Pulsar Functions for lightweight processing, and Pulsar SQL for querying topics. The catalog is smaller, the third-party integration surface is thinner, and the operational knowledge is concentrated in fewer people and a smaller set of vendors.

For an architecture decision this matters more than most Kafka vs Pulsar write-ups admit. The cost of an unfamiliar platform is not paid at selection time. It is paid at 3 a.m., eighteen months later, by whoever is on call.

• Choose Kafka when you want the largest ecosystem and hiring pool, when your workload is a straightforward high-throughput pipeline, when you want the fewest process types to operate, or when you need a managed service on any cloud without a third-party contract.
• Choose Pulsar when multi-tenancy is a product requirement rather than a nice-to-have, when you need built-in per-namespace geo-replication across several regions, when storage and compute genuinely need to scale independently, or when a single topic's data would not fit comfortably on one broker's disks.
• Do not migrate for elegance. Pulsar's architecture is cleaner in ways that are easy to admire on a whiteboard. If you cannot name the specific constraint it lifts for your workload, the migration will cost you tooling, retraining and institutional knowledge in exchange for a diagram you like better.

The honest summary: this is not a case where one system is behind. The gaps that used to make the Kafka vs Pulsar choice obvious in either direction — ZooKeeper, tiered storage, queue semantics — have all narrowed since 2024. What is left is a genuine architecture question about whether you need serving and storage to scale separately, and a genuine organizational question about which platform your team can actually run.

On storage, mostly yes. On tenancy, no.

Kafka's tiered storage moves closed log segments off local disk into object storage while the broker stays the only thing a client talks to. Consumers reading recent offsets hit the local hot set; consumers reading older offsets trigger a remote fetch that the broker streams back through the same fetch API. Local retention becomes a cache sizing decision rather than a capacity decision, and a partition can hold months of history without months of disk. How the local and remote retention settings interact is covered in our guide to Kafka retention policy.

Pulsar got here years earlier through BookKeeper offloaders and keeps two real advantages: offloading is a namespace policy rather than a per-topic setting, and because bookies were always the storage tier, broker disk was never in the picture. Kafka's implementation still has edges. Compacted topics cannot be tiered, cold catch-up reads pay remote fetch latency, and the remote storage plugin is something you choose and operate. For long retention on an ordinary event stream, the two are close enough that storage should not decide the platform.

Tenancy is different. In Pulsar a tenant owns namespaces, a namespace owns topics, and quotas, retention, offload, replication and permissions attach to the namespace. Kafka gets most of the way with conventions: a topic prefix per team, prefixed ACLs on that pattern, quotas per principal, and a separate cluster when one team is noisy enough to hurt the others. But it is discipline on top of a flat namespace, not a model the broker enforces. A mistyped ACL can open a prefix to the wrong team, and nothing rolls a team's settings into one object you can inspect.

Our verdict: Kafka has closed the storage gap and most of the operational gap. Pulsar still has the cleaner tenancy model, and the price is BookKeeper plus its metadata store running as a second stateful system beside the brokers.

In our engagements, far fewer organisations need first-class tenancy than ask about it. It matters when one platform team serves dozens of application teams with hard isolation, chargeback and self-service provisioning. It does not matter for a business with three or four producing teams in the same on-call rotation, which describes most of the enterprises we work with.

Almost every operational difference between the two comes back to where the bytes live.

A Kafka broker owns its partitions: log segments on its local disk, replicas on other brokers' local disks, KRaft controllers tracking who leads what. Storage capacity is broker capacity. When a disk fills or a broker runs hot, the fix is partition reassignment, which copies replica data across the network and waits for it to catch up. It is throttled, competes with live traffic, and takes hours on a large partition. Getting partition count and key distribution right up front, as we cover in Kafka partition strategy, is largely about avoiding that later.

Pulsar brokers hold no data. A topic is a managed ledger, each ledger is striped across an ensemble of bookies, and a new bookie is simply included in the next ledger. Scaling storage is adding bookies: nothing moves, no rebalance runs, and a broker failure is an ownership change measured in seconds rather than a replica catch-up.

The bill is a second stateful system with its own vocabulary. Every ledger carries an ensemble size, a write quorum and an ack quorum, and the relationship between the three sets both durability and how many bookies you can lose. When a bookie dies, autorecovery must detect under-replicated ledgers and re-replicate them; misconfigured or disabled, which we have found in clusters we inherited, the cluster runs quietly with less redundancy than anyone believes. Ledger recovery on failover has to fence the old owner and settle the last entry before the new broker serves reads.

The write paths differ in shape. A Kafka produce lands in the leader's page cache, fans out to followers, and is acknowledged once the in-sync replicas have it: one hop from the client, durability set by acks and min.insync.replicas. A Pulsar produce goes to the broker, which writes to the ensemble in parallel; each bookie fsyncs a journal entry before the ledger write, and the broker acknowledges at ack quorum. Two hops and an fsync on the hot path is why Pulsar's p99 write latency runs higher than Kafka's at equal durability.

Failure discovery follows the same split. A Kafka problem is usually an under-replicated partition or a broker that stopped leading, and every monitoring stack knows those metrics. Pulsar failures hide in the tier you are not watching: brokers healthy, bookies healthy, autorecovery silently behind.

That is the staffing reality. Engineers who have run Kafka in production are common. Engineers who can reason about BookKeeper quorums at 3am are rare. Our notes on Kafka on Kubernetes cover the storage settings that narrow the gap, but nothing closes it.

Pulsar's architecture earns its complexity when you must scale storage independently of serving, run tens of thousands of topics, or need namespace tenancy to avoid a cluster per team. Otherwise Kafka's simpler storage layer is the better trade, and we can help you design it or keep it running.

Evaluating a streaming platform, or supporting one you already run? AceMQ runs vendor-neutral assessments that start from the workload rather than the architecture diagram — see Kafka consulting and Apache Kafka support and consulting, or talk to an AceMQ engineer.

• Apache Pulsar — Architecture overview (brokers, BookKeeper, managed ledgers, metadata store)
• Apache Pulsar — Multi-tenancy
• Apache Pulsar — Geo-replication
• Apache Pulsar — Tiered storage
• Apache Kafka — 4.0.0 release announcement (KRaft-only, KIP-848 GA)
• Apache Kafka — 4.2.0 release announcement (share groups production-ready)
• Apache Kafka — Tiered storage