Ensuring messaging reliability for airport gate management systems worldwide
RabbitMQ
KubernetesQuorum Queues
Prometheus
GrafanaThe client achieved stable RabbitMQ operations on Kubernetes with proper resource isolation, automated recovery procedures, and comprehensive monitoring — ensuring reliable messaging for airport syste…
Overview
The company provides mission-critical airport gate management systems used worldwide. Their RabbitMQ deployment on Kubernetes was experiencing publish failures, cluster failover issues, and quorum queue synchronization problems that threatened system availability.
Challenge
RabbitMQ was co-located with PostgreSQL on shared Kubernetes nodes, causing resource contention. Quorum queue synchronization was failing at 3,000 IOPS storage, and cluster partitions were causing zombie bindings and message publishing failures. The team needed to upgrade to RabbitMQ 4.2 while maintaining stability across Azure and VMware environments.
Environment
Kubernetes clusters across Azure and VMware, RabbitMQ with PostgreSQL co-location, Prometheus/Grafana monitoring, C and Node.js client applications.
Approach
AceMQ conducted a multi-week intensive assessment with daily sessions, performing disk benchmarking with FIO, analyzing RAFT consensus logs, and designing infrastructure isolation strategies. The team provided comprehensive documentation covering failover procedures, monitoring, backup/restore, and federation.
Solution
- 1Separated RabbitMQ from PostgreSQL onto dedicated nodes with isolated SSD storage
- 2Recommended premium SSD V2 disks to achieve sub-millisecond latency required for quorum queues
- 3Developed automated detection and recovery scripts for stuck quorum queues and zombie bindings
- 4Configured Prometheus metrics and Grafana alerting for proactive cluster health monitoring
- 5Delivered comprehensive runbook for failover, backup/restore, and federation operations
- 6Guided upgrade path from RabbitMQ 3.x to 4.2 with stability validation
Outcome
The client achieved stable RabbitMQ operations on Kubernetes with proper resource isolation, automated recovery procedures, and comprehensive monitoring — ensuring reliable messaging for airport systems worldwide.
Technologies
RabbitMQKubernetesQuorum QueuesPrometheusGrafanaRelated Use Cases
RabbitMQ Kubernetes Stabilization
Hardening RabbitMQ in Kubernetes environments with StatefulSet tuning, quorum queue optimization, storage isolation, and memory/network configuration.
RabbitMQ Operational Visibility and Monitoring
Improving observability with Prometheus, Grafana, alerting, queue visibility, disk/memory thresholds, and retry metrics.
Ready for a RabbitMQ Health Check?
AceMQ's senior RabbitMQ engineers have handled this exact type of engagement before. Whether you need architectural guidance, hands-on remediation, or an ongoing managed partnership, we're ready to help.