Kafka

Exchanges vs Partitions: How RabbitMQ and Kafka Route Messages

A

AceMQ Engineering Team

Kafka Consulting & Support

Exchanges vs Partitions: How RabbitMQ and Kafka Route Messages

RabbitMQ decides where a message goes on the broker. Kafka decides in the producer. That one sentence explains most of the day-to-day differences between operating the two systems — where routing logic lives, what a new consumer costs you, and why one of them handles thousands of logical subjects comfortably and the other does not.

It also explains a difference that rarely appears in comparison tables: protocol reach. A broker that routes centrally can accept messages on several protocols and deliver them on others. A broker that treats the log as the interface cannot.

This is a companion to our RabbitMQ vs Kafka decision guide, going deep on the routing and protocol layer specifically.

The Worked Example: orders.#

Take a realistic requirement. Orders are published with subjects like orders.eu.created, orders.us.shipped, orders.apac.cancelled. An audit service needs every order event regardless of region or action.

Comparison showing RabbitMQ handling orders.# with a single topic exchange binding into one queue, against Kafka requiring separate topics or client-side filtering, and what happens when a new region is added

In RabbitMQ that is one topic exchange and one binding with the key orders.#. Every matching message lands in the audit queue. When the business opens a new region next quarter and orders.latam.created starts flowing, the audit service receives it automatically — the binding already matches. No producer change, no redeploy, nothing to coordinate.

In Kafka you choose between two imperfect options. Create a physical topic per subject and have the audit consumer subscribe to a regex pattern, which works but means the topic count grows with your business dimensions and every new subject is a provisioning step. Or publish everything to one orders topic and filter in the consumer, which means pulling every message across the network to discard the ones you do not want.

With three subjects, this is an argument about taste. With three thousand — per tenant, per region, per device class — it is the difference between a design that holds and one that becomes an operations problem.

Where the Logic Lives, and Who Pays for It

The deeper consequence is organisational, not technical.

When routing lives in the broker, changing message flow is a configuration change. Adding a consumer to an existing flow means declaring a queue and a binding. It happens at runtime, through the management UI or the HTTP API, and the publishing applications are not involved and do not need to know. A platform team can do it without a development team's release cycle.

When routing lives in the producer, changing message flow is a code change. The producer decides the partition, so any change to how messages are distributed is a pull request, a review, a build, and a deploy. That is fine when flows are stable. It is friction when they are not, and it means the messaging topology is coupled to your release cadence.

RabbitMQ also supports exchange-to-exchange bindings, which lets you build routing topologies — a fan-out that feeds several topic exchanges, for instance — without any of that structure existing in application code. There is no Kafka equivalent because there is nothing on the broker to compose.

The honest counterpoint: broker-side routing costs broker CPU, and a topology with thousands of bindings changing constantly is real load. We have seen environments where excessive binding churn combined with deeply layered wildcard keys became a genuine performance problem — usually a sign the subject hierarchy needs flattening rather than that routing is the wrong approach.

Protocol Reach: One Cluster, Many Clients

Because RabbitMQ routes centrally rather than treating the log as the interface, it can accept a message on one protocol and deliver it on another.

Protocol support matrix comparing RabbitMQ and Apache Kafka across AMQP 1.0, AMQP 0-9-1, MQTT, STOMP, JMS, stream protocol, Kafka protocol and WebSocket transports

This decides some architectures outright. If you have IoT devices speaking MQTT, legacy Java applications speaking JMS, and modern services speaking AMQP 1.0, RabbitMQ serves all three from one cluster — and it publishes exactly how message headers are converted when crossing between them, which is unusual enough to be worth calling out.

Kafka speaks the Kafka protocol. It is a genuinely good protocol and the de facto standard for streaming, but connecting anything else means running bridges or connectors as separate infrastructure to deploy, monitor, secure and patch. For a Kafka-native architecture that is a non-issue. For a mixed estate it is a standing operational cost that rarely appears in the initial comparison.

When Kafka's Model Is the Right One

Producer-side partitioning is not a limitation the Kafka team failed to fix. It is a deliberate trade that buys something real.

Partitioning is how a hot topic scales out. Because the producer chooses the partition, a single logical topic can be split across many brokers and consumed in parallel by a consumer group. A RabbitMQ quorum queue is not sharded — it tops out around 80,000 messages per second and you cannot split one queue across nodes to go faster. For a genuine single-stream firehose, Kafka's model wins and it is not close.

Keeping the broker simple keeps it predictable. A broker that does no routing work has fewer ways to become the bottleneck. Kafka brokers do less per message, and that is part of why the ceiling is where it is.

So the choice is not routing-versus-no-routing. It is whether your problem is many subjects with selective consumers — where broker-side routing is worth real money — or one enormous stream consumed broadly, where partitioning is worth more.

If you are trying to work out which of those describes your estate, talk to AceMQ. We run both.

FAQ

What is the difference between a RabbitMQ exchange and a Kafka topic?

An exchange is a routing component: publishers send to it and bindings decide which queues get a copy. A Kafka topic is a storage component: the producer picks the partition before sending. So routing logic lives on the broker in RabbitMQ and in application code in Kafka — a runtime binding change in one, a code change and redeploy in the other.

Does Kafka support AMQP or MQTT?

Not natively. Kafka speaks the Kafka protocol only, so connecting MQTT devices or JMS applications means running bridges or connectors as extra infrastructure. RabbitMQ serves AMQP 1.0, AMQP 0-9-1, MQTT, STOMP, JMS and its stream protocol against one cluster, with documented header conversion between them.

How do you handle a wildcard subject like orders.# in Kafka?

Either create a separate physical topic per subject and subscribe to a pattern, or publish everything to one topic and filter client-side, pulling data across the network to discard most of it. In RabbitMQ a topic exchange with a single orders.# binding handles it on the broker, and a new subject needs no producer change.

Why does routing matter more when you have many subjects?

Because the cost scales differently. With a dozen subjects, producer-side routing is a minor inconvenience. With thousands — per tenant, per region, per device class — Kafka forces a choice between an unmanageable topic count and heavy client-side filtering, while RabbitMQ handles it with bindings and cheap queues.

Can you change RabbitMQ routing without redeploying?

Yes. Bindings are runtime configuration, changeable through the management UI, HTTP API or CLI while the system runs, and publishers never know. Adding a consumer to an existing flow is a configuration task in RabbitMQ and a release in Kafka.

Does broker-side routing cost performance?

It costs broker CPU and saves network bandwidth and client work. Filtering at the broker means consumers receive what they asked for rather than everything, which for selective consumers is a large net win — RabbitMQ supports Bloom filters on streams for the same reason. The trade only reverses when nearly every consumer wants nearly every message.

Free Consultation

Get Expert Eyes on Your Kafka Cluster

Whether you're troubleshooting a production incident, planning a migration, or want a second opinion on your architecture — our team is ready. No pitch, just answers.

Email Us