Overview
Kong's rate limiting plugin defaults to a local counter policy, which keeps counts in each node's own memory. On a single node that is correct. On a horizontally scaled gateway it means every node enforces the full limit independently, so the effective ceiling is the configured limit multiplied by the number of nodes.
Challenge
Customers on a metered plan were consistently exceeding their documented request quota, and a partner integration overwhelmed a downstream service that the rate limit was supposed to protect. The limit looked correct in the plugin configuration. The gateway had been scaled from two to eight replicas some months earlier, and nobody revisited the counter policy.
Environment
Kong on Kubernetes with a horizontally scaled data plane, rate limiting applied per consumer, Redis available in the cluster.
Approach
AceMQ reproduced the overshoot with a controlled load test that pinned requests across replicas, confirming the effective limit scaled with replica count. We moved the counter store to a shared Redis backend and then load-tested the Redis path itself, since a shared counter adds a network hop into the request path and needs its own timeout and failure behavior defined.
Solution
- Reproduced quota overshoot with a load test distributed across gateway replicas
- Moved rate limit counters from the local policy to a shared Redis-backed store
- Defined Redis timeout and fault-tolerant behavior so a counter store outage degrades predictably
- Separated per-consumer, per-service, and global limits so one does not mask another
- Load-tested the shared-counter path to confirm added latency stayed within the gateway's budget
Outcome
Enforced limits now match configured limits regardless of replica count, and the downstream service that the limit exists to protect stopped being overwhelmed by single-partner traffic.
Technologies
Kong
Redis
Kubernetes
PostgreSQL
NGINX
Docker
