Moving to Cluster mode without breaking multi-key operations

Overview

A payments processor had scaled a standalone Redis instance to the largest available host and had nowhere left to go. Moving to Cluster mode was the obvious answer, but the application used multi-key operations and Lua scripts that assume all keys live on one node. AceMQ planned and validated the migration.

Challenge

Cluster mode rejects multi-key operations whose keys hash to different slots. Finding every such operation across a large codebase is harder than it sounds, since many are constructed dynamically. Hash tags can force related keys onto the same slot, but overusing them recreates the hot-node problem the migration exists to solve.

Environment

Redis on Kubernetes backing transaction deduplication, rate limiting, and idempotency key storage.

Approach

AceMQ inventoried the actual command mix from live traffic rather than from code review alone, which surfaced dynamically constructed multi-key operations that static analysis missed. Hash tag design was then constrained deliberately to preserve slot distribution, and the migration was staged with dual-read validation.

Solution

• Inventoried the live command mix to find multi-key operations and scripts that static code review missed
• Designed hash tags scoped narrowly enough to preserve slot distribution while keeping related keys co-located
• Rewrote the operations that could not be co-located to work across slots without correctness loss
• Validated client library cluster support and redirection handling ahead of any infrastructure change
• Designed a staged migration with dual-read validation and a defined rollback position at each phase
• Modeled slot distribution and per-node load against production key patterns before committing to the topology

Outcome

Technologies