Rate limits that let through N times the quota, where N is your node count

Overview

Kong's rate limiting plugin defaults to a local counter policy, which keeps counts in each node's own memory. On a single node that is correct. On a horizontally scaled gateway it means every node enforces the full limit independently, so the effective ceiling is the configured limit multiplied by the number of nodes.

Challenge

Customers on a metered plan were consistently exceeding their documented request quota, and a partner integration overwhelmed a downstream service that the rate limit was supposed to protect. The limit looked correct in the plugin configuration. The gateway had been scaled from two to eight replicas some months earlier, and nobody revisited the counter policy.

Environment

Kong on Kubernetes with a horizontally scaled data plane, rate limiting applied per consumer, Redis available in the cluster.

Approach

AceMQ reproduced the overshoot with a controlled load test that pinned requests across replicas, confirming the effective limit scaled with replica count. We moved the counter store to a shared Redis backend and then load-tested the Redis path itself, since a shared counter adds a network hop into the request path and needs its own timeout and failure behavior defined.

Solution

• Reproduced quota overshoot with a load test distributed across gateway replicas
• Moved rate limit counters from the local policy to a shared Redis-backed store
• Defined Redis timeout and fault-tolerant behavior so a counter store outage degrades predictably
• Separated per-consumer, per-service, and global limits so one does not mask another
• Load-tested the shared-counter path to confirm added latency stayed within the gateway's budget

Outcome

Technologies