Throughput went down when the platform scaled up

A transaction processing platform serving banking clients enabled Kafka and watched throughput drop by roughly a factor of seven. Adding service instances made no difference — performance stayed flat regardless of how many were running, which is the signature of a bottleneck rather than a capacity limit. A validation deadline meant the platform had to demonstrate 80,000 transactions per hour within a week.

Kafka 4.0 running in KRaft mode, underpinning a transaction processing platform for banking customers, with JMeter used for load validation.

Work focused on identifying why added instances produced no gain, then validating each change against a repeatable JMeter load profile so the improvement was measured rather than assumed. Failover behaviour was treated as a constraint on the tuning, not an afterthought.

• 1
  Established a repeatable load profile in JMeter so every configuration change could be measured against the same baseline
• 2
  Investigated why increased service instances produced constant throughput, isolating the serialisation point
• 3
  Tuned producer and broker configuration for the transaction pattern rather than for general-purpose throughput
• 4
  Validated failover recovery under load so resilience was not traded away for speed
• 5
  Reviewed KRaft-specific configuration against guidance written for ZooKeeper-era deployments