A Kafka rebalance is the process of redistributing partitions among consumers in a group. It is triggered whenever membership changes — a consumer joins, leaves, or is declared dead — or when the number of partitions on a subscribed topic changes. A healthy group rebalances occasionally and briefly. A group rebalancing every few minutes has a configuration problem, and the resulting pauses are usually mistaken for a broker or throughput issue.
Kafka rebalancing is not a fault in itself. It is how Apache Kafka distributes partitions to consumers and keeps work evenly spread as a deployment scales. Understanding how Kafka rebalancing works matters because rebalancing issues arise when it fires far more often than the workload warrants.
The mechanism is worth understanding, because almost every fix is a consumer configuration change rather than anything done to the cluster.
What Triggers a Kafka Rebalance?
Four things, and only the first two are intentional.
A new consumer joins the group. Scaling up, or a pod starting after a deployment. Expected — when a new consumer joins the group, the coordinator spreads the new partitions among the consumers already running.
A consumer leaves cleanly. A graceful shutdown that sends a leave request. Also expected. Whenever a consumer joins or leaves, redistribution follows.
A consumer is declared dead. This is where the trouble lives. The group coordinator initiates a rebalance the moment it decides a consumer is gone, which happens when either of two clocks runs out:
session.timeout.ms(default 45 seconds). The consumer sends heartbeats on a background thread everyheartbeat.interval.ms(default 3 seconds). Miss enough heartbeats and the coordinator evicts you. This catches genuine crashes, network partitions and long GC pauses.max.poll.interval.ms(default 5 minutes). The gap between successivepoll()calls on the main thread. Heartbeats keep flowing on the background thread, so the consumer looks alive — but if the application takes too long processing a batch, the coordinator evicts it anyway.
That second one causes most unexplained rebalances in production. The consumer has not crashed. It is just processing slowly, and Kafka cannot distinguish a slow consumer from a stuck one.
The number of partitions changes. When new partitions are added to a Kafka topic, reassignment follows. Subscription changes do the same.
The last two are rebalance events you would rather not have. Triggering a Kafka rebalance costs the same whatever the cause, and unnecessary rebalance activity is what turns a working consumer group into a stalled one.
What Actually Happens During a Kafka Rebalance?
All consumers in the group stop consuming, must rejoin the group, and wait before they can resume. The group leader — one of the consumers, not a Kafka broker — sends a request to the group coordinator, computes how partitions are assigned, and distributes the result. Partitions are revoked first, then handed back out — and which consumer ends up with specific partitions can change on every round.
The important property is that this is a synchronization barrier: all consumers in a consumer group move together, and the slowest member sets the pace. One consumer taking thirty seconds to rejoin means the whole group waits thirty seconds. No message processing happens during the rebalancing process, so a backlog builds and lag climbs even though every broker in the Kafka cluster is healthy.
This is why rebalances and lag are so often confused. If you are chasing growing lag rather than the rebalances causing it, diagnosing Kafka consumer lag covers the symptom side.
Eager versus cooperative rebalancing
The Kafka rebalance protocol has two generations, and which one you are running changes the cost of every rebalance enormously.
Eager rebalancing is the original, stop-the-world behavior. Every consumer revokes *all* of its assigned partitions before the new assignment is computed. Even if the reassignment only moves two partitions out of two hundred, everything stops. The pause scales with group size and rejoin time, not with how much actually changed.
Incremental rebalancing revokes only the partitions that are actually moving. Consumers keep processing everything they retain. The cooperative rebalancing protocol runs as two lighter rounds instead of one heavy one, so the impact of rebalancing scales with the size of the change rather than with how many members there are. This incremental cooperative approach is what most estates should be running.
This has been available since Kafka 2.4 and is not always the default in an older Kafka client. Static membership arrived slightly earlier, in Kafka 2.3. Switching is one line:
partition.assignment.strategy=org.apache.kafka.clients.consumer.CooperativeStickyAssignor⚠️ Do not just set this and deploy. Moving from eager to cooperative requires a two-phase rolling upgrade — first deploy with both strategies listed so the group can agree, then remove the eager one on a second pass. Changing it in one step on a running group will fail the rebalance.
Newer releases go further again. The new consumer rebalance protocol moves assignment computation to the broker and removes the global synchronization barrier entirely, so consumers no longer all stop together. This new protocol reached general availability in Apache Kafka 4.0. If you are on a recent release it is worth evaluating — but most production estates are not there yet, and the sticky assignor delivers the majority of the benefit today.
Partition assignment strategies
| Assignor | Behavior | Rebalance cost |
|---|---|---|
RangeAssignor | Per-topic contiguous ranges | Eager; can distribute unevenly across multiple topics |
RoundRobinAssignor | Even spread across all partitions | Eager; reassigns everything |
StickyAssignor | Even spread, preserves prior assignment where possible | Eager, but moves fewer partitions |
CooperativeStickyAssignor | Sticky, and revokes only what moves | Incremental — the one to want |
Stickiness matters beyond the pause itself. When a consumer keeps the same topic partitions across a rebalance, it keeps warm local state — caches, in-flight aggregations, Kafka Streams state stores. Reshuffling needlessly throws that away, and the recovery cost often exceeds the rebalance itself.
If you use Spring Kafka or another framework wrapper, this is set through the same underlying property — the framework does not change which strategies are available to you.
The Kafka Rebalance Storm: Why It Never Recovers
The rebalancing problems that bring people to us almost always follow one shape:
- Processing slows down — a downstream API degrades, or a batch gets larger.
- A consumer exceeds
max.poll.interval.msand is evicted. - The group rebalances. Everyone stops.
- The backlog grows during the pause.
- On resume, each consumer pulls a full batch against a bigger backlog and takes *longer* to process it.
- Another consumer exceeds the interval. Return to step 3.
Data processing throughput collapses while the cluster itself stays healthy.
It never recovers on its own, because each new Kafka rebalance makes the next one more likely — which is exactly why teams start investigating the cluster rather than how they configure the consumer.
Breaking the loop is usually a matter of making batches finish inside the interval:
- Reduce
max.poll.records. The most effective single change. Fewer records per poll means less time between polls. Start by halving it. - Raise
max.poll.interval.msto genuinely exceed worst-case processing time, not typical time. Measure the p99, do not estimate. - Move slow work off the polling thread. If processing involves a network call of unpredictable duration, the poll loop is the wrong place for it.
- Do not increase
session.timeout.msto fix this. It is a different clock and will not help — it only delays detection of genuinely dead consumers.
Static membership
Where consumers restart routinely — rolling deployments on Kubernetes, most obviously — this stops the restart from triggering one at all.
Assign each consumer a stable group.instance.id. On restart inside the timeout window, the coordinator recognizes the returning member and hands back its previous partitions without redistributing anything:
group.instance.id=consumer-3
session.timeout.ms=45000The trade-off is honest: a genuinely dead member is not replaced until its session times out, so its partitions sit unconsumed for that window. Decide whether rebalancing is necessary on restart before enabling it. Set the timeout to comfortably exceed your rolling restart time and no more.
There is also a broker-side setting worth knowing for startup. group.initial.rebalance.delay.ms (default 3 seconds) makes the coordinator wait briefly before the first rebalance, so consumers starting together form one group rather than rebalancing repeatedly as each arrives. It is a small thing that removes a lot of noise from Kubernetes deployments.
What to monitor
Watch three things, and watch them together:
- Rebalance rate. The signal itself. Anything above occasional deserves investigation — and a consumer stuck in a rebalance loop shows up here first.
- Rebalance duration. Rising duration means consumers are slow to rejoin — often the same slow processing that triggered the rebalance.
- Consumer lag during rebalances. Confirms whether pauses are actually costing you throughput or are short enough not to matter.
A consumer group that runs a Kafka rebalance a few times a day around deployments is fine. One rebalancing every few minutes is telling you max.poll.interval.ms is being exceeded, and no amount of cluster tuning will change that.
Seeing rebalance storms or unexplained consumer pauses? These almost always sit in consumer
configuration rather than the cluster, which is why broker metrics look clean while throughput
collapses. AceMQ runs structured Kafka assessments that find them —
If one person carries all of this knowledge, that is a staffing risk rather than a Kafka one: what self-supporting Kafka actually costs.
Cooperative rebalancing in practice: verifying it, migrating to it, and what Streams and Connect do
Range, round-robin and the original sticky assignor all use the eager protocol. On any membership change the coordinator moves the group to PreparingRebalance, every consumer revokes every partition it owns, everyone rejoins, and the leader hands out a fresh assignment. The whole group stops for the round trip.
Kafka 2.4 introduced cooperative rebalancing for consumers. Under the incremental cooperative protocol a rebalance runs in two phases. In the first, the leader computes the new assignment and each consumer revokes only the partitions leaving it, while it keeps polling and committing on everything else. A second rebalance immediately follows, in which the freed partitions go to their new owners. The only client-side assignor that uses this protocol is CooperativeStickyAssignor.
Most groups we get called about are running eager without knowing it. First, the consumer config: partition.assignment.strategy is a list, and on recent clients the default list contains both range and cooperative-sticky. The coordinator selects one strategy that every member supports, so the default lands on range. Second, describe the group and look at what the coordinator actually selected:
kafka-consumer-groups.sh --bootstrap-server broker:9092 \
--describe --group orders-svc --stateThe ASSIGNMENT-STRATEGY column reads range or cooperative-sticky. Range means every restart is still stop-the-world.
Migrating a live group takes two rolling bounces, and the order matters. On the first, set partition.assignment.strategy to the current eager assignor followed by CooperativeStickyAssignor, keeping the eager one first so it stays selected while old and new instances are mixed. On the second, remove the eager assignor. Doing it in one step fails because a member joining mid-bounce finds no strategy in common with the others and the rebalance errors out.
Kafka Streams applications already run the cooperative protocol through their own assignor, so nothing needs configuring there. Kafka Connect workers balance connectors and tasks with a separate incremental cooperative protocol controlled by connect.protocol, and the default enables it when every worker supports it.
The trade-off is small. A cooperative rebalance takes slightly longer end to end because it is two rebalances rather than one, and the partitions that move are unassigned for that window. In exchange, the partitions that do not move never stop. The partition count you choose bounds how many consumers can share the work, which we cover in our partition strategy guide.
Fixing "the group is rebalancing, so a rejoin is needed" and "preparing rebalance" loops
"The group is rebalancing, so a rejoin is needed" is the message attached to the REBALANCE_IN_PROGRESS error code. The coordinator returns it to a consumer whose request carries a generation that is no longer current, most often a heartbeat, sync or offset commit sent after the group moved to PreparingRebalance. On its own it is not a fault; the client rejoins and the group settles. It becomes a problem when it repeats every few seconds, the group never reaches Stable, and lag climbs. The coordinator broker logs a matching "Preparing to rebalance group ... (reason: ...)" line, and that reason field is the fastest way to tell which of the following is happening.
1. max.poll.interval.ms exceeded. The consumer is alive but took too long between polls, so it was dropped. Confirm: consumer logs say "consumer poll timeout has expired" and the broker reason cites a member leaving. Fix: lower max.poll.records until worst-case batch processing fits inside the interval, then raise max.poll.interval.ms only if the p99 still does not fit.
2. session.timeout.ms too low for GC pauses or network. The heartbeat thread stalls or its packets are delayed and the member is evicted. Confirm: the broker reason says "heartbeat expiration", GC logs show pauses near the timeout, and there is no poll timeout message. Fix: raise session.timeout.ms within the broker's group.max.session.timeout.ms, and fix the pause. Producers in the same JVM show related symptoms, covered in producer timeout causes and fixes.
- 3. Consumers scaling in a loop. An autoscaler adds replicas when lag rises, lag falls, it removes them, lag rises again. Every scale event is a rebalance. Confirm: describing the group with
--membersshows a member count that changes minute to minute and lines up with deployment events. Fix: add a scale-down stabilisation window, cap replicas at the partition count, and scale on a smoother signal than raw lag. Measuring lag properly is in our consumer lag guide. - 4. Static membership not in use. Without
group.instance.id, a rolling deploy of ten pods is ten leaves and ten joins. With it, a pod returning inside the session timeout resumes its old assignment with no rebalance. Confirm:--members --verboseshows generated member ids that change on every restart. Fix: set a stable instance id per pod. - 5. Coordinator or broker change. The
__consumer_offsetspartition hosting the group moved brokers, the group was reloaded, and every member had to rediscover and rejoin. Confirm: broker logs show "Loading group metadata" on a new broker and the timing matches a restart or leader election. Fix: nothing on the client. One rebalance per broker bounce is expected; repeated coordinator moves mean the cluster itself is unstable.
The three properties that end most of these loops:
max.poll.records=100
group.instance.id=orders-svc-${POD_ORDINAL}
partition.assignment.strategy=org.apache.kafka.clients.consumer.CooperativeStickyAssignorWe apply these fixes for enterprise clients under 24/7 Kafka support with a 15-minute P1 response, and we design groups this way from the start on Kafka consulting engagements.
This is step four of ten in the Kafka operations guide, which takes the operational decisions in the order they arrive on a production estate.
FAQ
What triggers a Kafka consumer group rebalance?
A consumer joining, leaving cleanly, or being declared dead by the group coordinator — either through missed heartbeats past session.timeout.ms or through exceeding max.poll.interval.ms between polls. A change in the partition count of a subscribed topic also triggers one.
Why does rebalancing cause latency and backlog?
Because it is a synchronization barrier. Consumers stop consuming while the group rejoins and waits for a new assignment, so the slowest member sets the pause for everyone. Nothing is processed during that window and lag accumulates.
What is cooperative rebalancing in Kafka?
An incremental protocol that revokes only the partitions actually changing hands, letting consumers keep processing everything else. Eager rebalancing revokes every partition from every consumer regardless of how little is moving.
How do I stop frequent Kafka rebalances?
Find out which clock is expiring. If batches take too long, lower max.poll.records and raise max.poll.interval.ms past the worst case. If restarts are the cause, use static membership. Then move to the cooperative sticky assignor to make the remaining rebalances cheaper.
What is static group membership?
A stable group.instance.id per consumer that lets the coordinator recognize a restarting member and return its previous partitions without a rebalance, provided it returns inside the session timeout.
Which partition assignment strategy should I use?
CooperativeStickyAssignor for most workloads — it minimizes both partition movement and rebalance disruption. Migrate to it over two rolling deployments rather than changing the setting in one step.
Does adding partitions trigger a Kafka rebalance?
Yes. Increasing the partition count on a subscribed topic causes the group to reassign, so treat partition changes as a scheduled operation rather than a routine one.
Is a Kafka rebalance the same as reassigning partitions across brokers?
No, and the shared vocabulary causes real confusion. A consumer group rebalance redistributes partitions among *consumers*. Moving partition replicas between *brokers* is partition reassignment — a separate cluster operation with different tooling and different risks.
Why does my Kafka consumer group keep rebalancing?
In our experience the causes, in order, are: processing a poll batch takes longer than max.poll.interval.ms; session.timeout.ms is too short for the JVM's GC pauses or the network; an autoscaler is adding and removing consumers in a loop; the consumers restart without a group.instance.id, so every deploy is a leave plus a join; or the group coordinator moved brokers. The reason field in the coordinator broker's "Preparing to rebalance group" log line tells you which. Describe the group with --state to see whether it ever reaches Stable.
How do I stop a Kafka rebalance from causing downtime?
Three settings remove most of the pain. Switch the group to CooperativeStickyAssignor so only moving partitions pause, using the two-bounce migration. Give each consumer a stable group.instance.id so restarts within the session timeout do not trigger a rebalance at all. Cap max.poll.records so a slow batch cannot get a healthy consumer evicted. Then stop the churn at the source: stabilise autoscaling and keep the replica count at or below the partition count. A well-configured group should rebalance only when you deliberately change its size.
