A cache design where invalidation is a decision, not an accident

Overview

A payments processor expanding into new regions needed a caching design that would not produce stale reads across regions. AceMQ designed the topology, namespacing, and invalidation strategy before the expansion began.

Challenge

The existing single-region setup relied on individual services deleting keys after writes, with each service using its own key naming. Nobody had a full picture of which keys a given write should invalidate, so stale reads happened and were resolved by shortening TTLs — which pushed load back onto the database. Extending that model across regions would multiply the problem, since a write in one region had no path to invalidate a cached copy in another.

Environment

Multi-region AWS deployment on Kubernetes, latency-sensitive payment authorization paths, regional data residency constraints.

Approach

We designed the cache around explicit ownership: each key namespace has exactly one owning service responsible for its lifecycle. Rather than distributing deletes across regions, the design uses version-keyed namespaces so an invalidation is a single version bump that makes stale entries unreachable and lets them age out naturally.

Solution

• Key namespace scheme with a single owning service per namespace, ending shared implicit ownership
• Version-keyed namespaces so invalidation is one version increment rather than an enumerated set of deletes
• Region-local cache tiers with residency boundaries respected, and explicit rules for what is never cached cross-region
• TTL strategy per data class based on tolerable staleness, replacing the blanket short TTLs used to mask invalidation bugs
• Failure behavior defined so a cache outage degrades to direct reads with concurrency limits rather than a database stampede
• Guidance on where a durable store is the better fit than a volatile cache, keeping Memcached for the workloads it suits

Outcome

Technologies