Strengthening payment infrastructure resilience for a global financial services company

Overview

The company operates a global payments platform built on microservices with RabbitMQ as the core messaging backbone. With over 200 production changes weekly and strict availability requirements, any RabbitMQ instability directly impacts payment processing.

Challenge

A network outage in AWS triggered incorrect message bindings and delivery failures across availability zones. Messages were routing incorrectly due to communication failures between AZs, causing duplicate processing and delivery failures. The team needed to migrate from classic mirrored queues to quorum queues for RabbitMQ 4.x compatibility while maintaining zero-downtime operations.

Environment

AWS multi-AZ deployment, RabbitMQ 3.13 with migration path to 4.1, Spring AMQP microservices architecture, Kubernetes orchestration.

Approach

AceMQ provided a comprehensive assessment of RabbitMQ production definitions, identified policy and regex errors, and delivered best-practice documentation for Spring Boot AMQP usage including concurrency, bindings, policies, and cluster partition handling.

Solution

• Diagnosed root cause of AWS network outage impact on RabbitMQ cluster partition behavior
• Recommended migration from 'auto heal' to 'pause minority' partition handling to prevent data loss
• Implemented publisher confirms and event listeners for retry logic in Spring AMQP clients
• Designed dynamic queue management strategy to prevent resource leaks in microservices
• Planned migration path from classic mirrored queues to quorum queues for RabbitMQ 4.x

Outcome

Technologies