Overview
A typical engagement starts with a web platform that keeps user sessions in Valkey. Memory is close to the limit at peak, some users are logged out without explanation, and the team does not know whether the cache is evicting keys or refusing writes.
Challenge
At maxmemory, Valkey's behavior is set by the eviction policy. With noeviction, writes are refused with an error and reads continue; with an eviction policy, keys are removed, and a policy that does not match how sessions expire can remove live ones. Memory the dataset does not account for also matters: fragmentation, replication buffers and the copy-on-write memory used while a snapshot is written all come out of the same host.
Environment
A primary with two replicas on Linux virtual machines, snapshot persistence, session keys with and without an expiry set, and a few large keys written by a reporting job.
Approach
AceMQ engineers read INFO memory and INFO stats over a week of traffic, compare used_memory, used_memory_rss and the fragmentation ratio, check evicted_keys, and run valkey-cli --bigkeys and --memkeys to find the largest keys. They test how much extra memory a snapshot needs while it is written and how a full resynchronization affects the primary. The recommendation sets maxmemory, the eviction policy and expiry rules from the measurements.
Solution
- Memory trend and fragmentation ratio over a week of traffic
- Large keys and keys with no expiry identified, with the job that writes them
- maxmemory and eviction policy chosen from how sessions are created and expire
- Headroom for snapshot and replication buffers measured, not assumed
- Replication backlog sized so a short disconnect does not force a full resynchronization
- Alerts on memory, evictions and rejected writes
Outcome
The engagement delivers measurements, a recommended maxmemory and eviction policy with reasons, expiry rules for the keys that lack them, a sizing note for the next growth step and alert definitions. The team applies the changes, or AceMQ engineers stage them one node at a time. Application changes, such as adding expiry to keys, stay with the customer's developers.