Throttle limits that hold on one node and not the next
WSO2
KubernetesKafka
PostgreSQL
DockerEnforced throttling matched published subscription tiers across all gateway nodes, and adding gateway capacity no longer risks reintroducing the divergence.
Overview
WSO2 API Manager can evaluate throttling either locally per gateway node or centrally through the traffic manager. When gateway nodes lose their connection to the traffic manager, or when the event topics between them are misconfigured, throttling decisions diverge and the enforced limit stops matching the published policy.
Challenge
Consumers on a metered tier were exceeding their subscription limits, but only intermittently and not reproducibly on any single node. Gateway nodes had been scaled up, and the new nodes' traffic manager event receiver configuration did not match the originals, so throttle decision events were not reaching all of them. Under load, the divergence widened.
Environment
WSO2 API Manager on Kubernetes with multiple gateway nodes, a traffic manager tier, and a shared user store.
Approach
AceMQ traced throttle decision events from the traffic manager through to each gateway node and identified the nodes that were not receiving them. We corrected the event receiver configuration, then validated throttle consistency under load by driving traffic deliberately at each node.
Solution
- 1Traced throttle decision event flow from traffic manager to each gateway node to find the gaps
- 2Corrected event receiver and topic configuration on gateway nodes added during scale-out
- 3Aligned throttle policy tiers and key templates so policies evaluate identically across nodes
- 4Added failure behavior for gateway nodes that lose traffic manager connectivity
- 5Validated enforced limits under load with traffic driven at each node individually
- 6Templated gateway node configuration so future scale-out cannot drift
Outcome
Enforced throttling matched published subscription tiers across all gateway nodes, and adding gateway capacity no longer risks reintroducing the divergence.
Technologies
WSO2KubernetesKafkaPostgreSQLDockerRelated Use Cases
WSO2 Registry and Database Deadlock Support
Resolving registry database deadlocks under concurrent load that intermittently froze WSO2 API deployment and gateway startup.
WSO2 Platform Architecture Assessment
Reviewing a WSO2 deployment's topology, database layout, high availability posture, and gateway sizing against its actual traffic profile.
WSO2 API Manager Upgrade and Migration Consulting
Planning a multi-version WSO2 API Manager upgrade including registry migration, API redeployment, and identity integration changes.
Docker OOMKilled Container Remediation
Resolving containers repeatedly OOMKilled because the JVM and Node runtimes inside them were sizing heap against host memory rather than the cgroup limit.
Kong 502 and Upstream Health Check Remediation
Tracing intermittent 502s at the Kong gateway to misconfigured active health checks and stale DNS resolution of upstream service names.
Airbyte Deployment Hardening and Operations Consulting
Taking a proof-of-concept Airbyte install to a production-grade deployment with proper isolation, secrets handling, resource limits, and recovery procedures.
Facing a WSO2 Production Issue?
AceMQ's senior WSO2 engineers have handled this exact type of engagement before. Whether you need architectural guidance, hands-on remediation, or an ongoing managed partnership, we're ready to help.