RemediationSoftware / Digital PlatformsCloud

Memory growth that looked like a workload problem turned out to be an indexing bug

Digital Collaboration Platform

Valkey

Redis

Kubernetes

Result

Memory growth stopped being an incident. The cluster held steady between maintenance windows, the team gained a documented mitigation they could run themselves, and the Valkey migration proceeded as a…

Overview

A collaboration platform ran Redis as the session and search tier behind a customer-facing product. Memory climbed steadily between restarts until the cluster crashed, and the pattern had survived several rounds of resizing.

Challenge

Adding memory did not help, because the growth was not proportional to traffic. The cluster also carried a memory ceiling that had been set without units, so the intended limit was never actually applied. Restarts recovered memory but dropped sessions.

Environment

Containerised Redis with replicas and Sentinel-based failover, running in a managed cloud environment behind a high-frequency write workload.

Approach

AceMQ engineers reproduced the growth in a CI cluster, isolated it to the full-text index by dropping and rebuilding the index and watching memory return, and confirmed the behaviour matched a known defect in the search module version in use. A staggered restart of replicas before the master gave the team a safe interim mitigation while the upgrade path was evaluated.

Solution

1
Reproduced memory growth in a non-production cluster to isolate the cause
2
Traced the leak to the full-text search module rather than the application workload
3
Corrected a memory ceiling that had been configured without units
4
Set an explicit eviction policy so the cluster degrades instead of crashing
5
Introduced staggered replica-then-master restarts to avoid failover risk
6
Evaluated Valkey as the supported long-term path, with feature-flagged rollout

Outcome

Technologies

Valkey

Redis

Kubernetes

Related Use Cases

Assessment

Redis Cluster Health and Architecture Assessment

AceMQ's Redis Health and Architecture Assessment identifies cluster vulnerabilities, performance bottlenecks, and optimization opportunities before they become production incidents — delivering a prioritized remediation roadmap.

Facing a Valkey Production Issue?

AceMQ's senior Valkey engineers have handled this exact type of engagement before. Whether you need architectural guidance, hands-on remediation, or an ongoing managed partnership, we're ready to help.