24/7 Redis support from senior engineers, no tier-1 triage

Overview

A healthcare analytics provider ran a single Redis instance holding both cache entries and user session state. When memory pressure hit, the eviction policy removed keys without regard to which were disposable, so users were logged out mid-workflow. AceMQ took over Redis support with named senior engineers.

Challenge

Mixing cache and session data in one keyspace under a single eviction policy means every policy choice is wrong for one of the workloads. Compounding this, many cache keys carried no TTL at all, so a volatile eviction policy had nothing eligible to evict and the instance returned out-of-memory errors instead. Memory reporting was also being misread, with fragmentation mistaken for data growth.

Environment

Redis on Azure supporting session state and query result caching for a clinical analytics application.

Approach

AceMQ separated the workloads first, because no single eviction policy serves both, then corrected the TTL and policy configuration for each. Memory reporting was clarified so the operations team could distinguish data growth, fragmentation, and replication buffer usage.

Solution

• Separated session state and cache into distinct instances with policies appropriate to each workload
• Applied TTLs consistently across cache keys so volatile eviction policies have eligible candidates
• Set maxmemory and eviction policy per instance against measured working set rather than host memory
• Clarified memory reporting so fragmentation, data, and replication buffers are read correctly
• Identified and remediated the key patterns driving unbounded growth in the cache keyspace
• Established 24/7 escalation to senior Redis engineers with a 15-minute emergency response commitment

Outcome

Technologies