Mosquitto vs Kafka: MQTT Broker or Event Log?

Mosquitto implements MQTT, a publish-subscribe protocol designed for constrained devices and unreliable links. A client connects, publishes to a topic such as plant1/line3/temp, and the broker forwards the message to every client whose subscription matches, including wildcards like plant1/+/temp. MQTT carries the features devices need: a keepalive so the broker notices dead connections, a last will message the broker publishes when a client drops without disconnecting, retained messages so a new subscriber immediately gets the last value on a topic, and persistent sessions that hold QoS 1 and 2 messages for a client while it is offline. Mosquitto is small enough for a single-board computer as well as a server.

Kafka is built for services, not devices. Producers append records to topics split into partitions; consumers pull, track their own offsets and can go back to any retained position. Reading does not delete anything, so a fraud model, a data lake loader and a dashboard can each consume the same telemetry independently, and a new one can start from last month. That is the job Mosquitto does not attempt.

MQTT defines three quality-of-service levels: 0 (at most once), 1 (at least once, so duplicates are possible) and 2 (exactly once, using a four-step handshake). Each applies to one hop, publisher to broker or broker to subscriber, and the MQTT 5.0 specification has the broker deliver at the lower of the publish QoS and the QoS granted to the subscription. A device publishing at QoS 2 to a subscriber at QoS 0 gets QoS 0 on the second hop. QoS 2 does not make delivery into Kafka exactly-once either: the connector reading from the broker is one more hop with its own guarantees.

Kafka producers default to acks=all with idempotence enabled, which removes duplicates from producer retries, and transactions give exactly-once processing between Kafka topics. In practice an MQTT-to-Kafka pipeline is at-least-once end to end, so give each reading an ID or a device timestamp and make consumers idempotent.

Mosquitto keeps its state in memory. With persistence true (off by default) it writes connection, subscription and message data to mosquitto.db on shutdown and every autosave_interval, 1,800 seconds by default, so a crash loses what changed since the last save. Mosquitto 2.1 added a persist-sqlite plugin, and the documentation now recommends plugin-based persistence over the built-in file. Queues for offline clients are bounded: once a client has max_queued_messages waiting, 1,000 by default, further messages for it are dropped. None of this is history; a message delivered is gone.

Kafka writes every record to disk on each replica and keeps it for the topic's retention, seven days by default (log.retention.hours=168), by size, forever, or as the latest value per key with log compaction. That is why telemetry that must be reprocessed, audited or used to train models belongs in Kafka, or in storage fed from it, rather than in the MQTT broker.

An open-source Mosquitto broker is a single process. To spread load or link sites you run several and connect them with bridges: a bridge is a client connection from one broker to another, configured with connection, address and one or more topic lines giving the pattern, the direction (in, out or both), the QoS and optional prefixes for remapping. Two defaults catch people out: a bridge topic's QoS defaults to 0, and bridge_protocol_version defaults to MQTT 3.1.1, with mqttv50 available. Clustering and high availability are not in the open-source edition; the Mosquitto site lists them for Cedalo's commercial Pro Edition.

Kafka scales a topic by adding partitions across brokers and replicates each partition, usually three times, so a broker can fail without losing acknowledged data. Since Kafka 4.0, released on 18 March 2025, KRaft controllers manage metadata and ZooKeeper is gone. Our Kafka architecture guide covers how that fits together.

Most IoT platforms combine the two. Devices publish to Mosquitto, at each site or centrally; a Kafka Connect MQTT source connector subscribes to the broker and writes the messages to Kafka topics; stream processors, databases and analytics read from Kafka. Commands travel the other way through an MQTT sink connector. Two connector families are common: Confluent's MQTT source and sink connectors, which run for a 30-day trial and then need a Confluent enterprise subscription, and the Lenses Stream Reactor MQTT source and sink, which are Apache 2.0.

Points that decide whether the bridge loses data:

• Subscribe the connector at QoS 1 with a persistent session, so the broker queues messages while the connector restarts, and size max_queued_messages for the longest restart you expect.
• Map many MQTT topics to a few Kafka topics, and put the device ID in the Kafka record key so each device's readings stay in order in one partition. A Kafka topic per device does not scale.
• Expect duplicates at QoS 1 and deduplicate downstream.
• If edge brokers bridge to a central broker first, set the bridge topic QoS explicitly instead of leaving it at 0.

If you want MQTT and service messaging in one broker, RabbitMQ is the usual alternative to Mosquitto. Its MQTT plugin ships with the core distribution and supports MQTT 3.1, 3.1.1 and 5.0. Messages from MQTT clients are routed through the amq.topic exchange, so AMQP 0.9.1 and AMQP 1.0 consumers can read them from bound queues, and MQTT publishers can also write to RabbitMQ streams. Durable subscriptions can use quorum queues (mqtt.durable_queue_type = quorum), and RabbitMQ clusters natively, which the open-source Mosquitto broker does not. Kafka still sits behind it when the data needs long retention and many independent consumers. Our RabbitMQ consulting team designs these deployments, and RabbitMQ vs Kafka compares the two brokers directly.

1. Are the clients devices on constrained hardware or unreliable networks? An MQTT broker such as Mosquitto.
2. Must the data be stored, replayed or read by many independent systems? Kafka.
3. Do you need high availability for the MQTT layer without a commercial license? RabbitMQ's MQTT plugin, or another clustered MQTT broker.
4. Do you need both? Mosquitto or RabbitMQ at the edge, a Kafka Connect MQTT connector, and Kafka as the system of record.