Modernizing real-time data flows across six manufacturing plants

Overview

The company operates multiple manufacturing plants that rely on real-time data ingestion to drive production decisions. Their existing system used SQL Server triggers to capture database events and relay them downstream, but this approach was creating substantial CPU overload on production database servers and threatening operational stability.

Challenge

The SQL-trigger-based ingestion system was causing critical CPU contention on production databases, impacting real-time data delivery. Manufacturing operations require low-latency data flows — delays of more than a few minutes can trigger plant-floor alarms and even mandatory shutdowns. The existing worker middleware was difficult to maintain and lacked vendor accountability.

Environment

On-premises Rancher Kubernetes cluster, SQL Server databases, Ignition SCADA, multiple manufacturing execution systems across six plants, with a production rollout timeline extending to 2027.

Approach

AceMQ engaged in a phased consulting engagement beginning with a three-week discovery and architecture assessment. The team evaluated multiple messaging tools including RabbitMQ, MQTT, and Kafka, then designed and built a proof-of-concept demonstrating a shadow table pattern for non-blocking data extraction that eliminates database contention while maintaining real-time delivery.

Solution

• Designed a shadow table pattern enabling non-blocking data extraction from SQL Server without impacting production database performance
• Built an Ignition-to-RabbitMQ data flow demo to validate real-time event publishing
• Evaluated and recommended messaging tools (MQTT, Kafka, RabbitMQ) for different use cases across the plant floor
• Created an architecture blueprint for event-driven data ingestion across six plants
• Provided ongoing vendor-accountable support and maintenance strategy

Outcome

Technologies