Checking whether more cache memory would fix anything before buying it

Overview

A SaaS provider was about to double their Memcached footprint to improve a disappointing hit rate. AceMQ assessed the tier first to determine whether capacity was actually the constraint.

Challenge

Aggregate hit rate looked poor, which made more memory the obvious answer. But the aggregate hid two separate problems. A meaningful share of gets were for keys that are only ever read once — cache misses by design, which no amount of memory fixes. And key distribution across the ring was badly uneven because a hashing configuration difference between two client libraries meant some nodes were near capacity and evicting while others sat half empty.

Environment

Memcached tier on AWS accessed by services in two languages using different client libraries.

Approach

The assessment breaks hit rate down by key pattern instead of treating it as a single number, and measures actual key and memory distribution per node. That separates the genuinely uncacheable traffic from the workload that would benefit from more memory, and exposes distribution problems that adding nodes would not fix.

Solution

• Hit rate decomposed by key prefix and access pattern, separating single-read keys from genuinely reusable ones
• Per-node key count and memory distribution measured to expose ring imbalance
• Client library hashing configuration compared across both languages, identifying the mismatch causing the imbalance
• Working set size estimated per key class so capacity requirements rest on measurement rather than a doubling guess
• Uncacheable access patterns identified and returned to the application teams as candidates for removal from the cache path
• Sizing recommendation with expected hit rate outcome per option, including the option of adding no capacity at all

Outcome

Technologies