Back to all use cases

Architecture AdvisoryCross-Industry

Decide what the streaming platform is made of, and who runs it, before the third team builds its own

Regional Insurer

What a platform review covers when several teams want streaming and nobody has decided which broker, processor and operating model the company will run.

Overview

A typical review starts when two or three teams each want real-time data and have started on different tools. AceMQ reads what runs today, what each team needs, and what the organisation can operate, and returns a written choice of platform parts and an operating model.

Challenge

Each team has chosen for its own use case: one on a Kafka cluster still in ZooKeeper mode, one evaluating Flink for stateful processing, one asking for change events from a relational database into a dashboard. There is no owner for upgrades, no agreed alerts on lag or partition health, and no view of which workloads need a streaming platform at all.

Environment

A mix of self-run Apache Kafka on virtual machines, a managed streaming service in one cloud, relational databases as change sources, and a warehouse as one downstream consumer.

Approach

AceMQ builds a workload inventory, compares the options against it, drafts a reference architecture and an upgrade calendar against the vendors' published support dates, and agrees an operating model with the platform team, from escalation cover to a managed tier.

Solution

  1. Workload inventory: sources, consumers, latency needs, state and delivery guarantees for each use case
  2. A written platform choice covering broker, stream processor, connectors and change data capture, including the workloads better served without a streaming platform
  3. A reference architecture for topic and partition design, retention, security and where processing runs
  4. A version plan listing the KRaft, Java and connector prerequisites for moving to a supported Kafka release
  5. Alerts and runbook steps for consumer lag, partition and controller health, disk, checkpoints and connector state
  6. An operating model that names who holds the pager and which of the four support tiers applies

Outcome

The customer ends up with one written platform decision instead of three tool choices, a reference architecture its engineers have reviewed, an upgrade calendar and an agreed operating model with a named owner for each stage.

Technologies

KafkaApache FlinkDebezium
  • Assessment

    Kafka Architecture Assessment and Migration Advisory

    Enterprise OrganizationsCross-Industry

    AceMQ's Kafka assessment service evaluates streaming architecture health, identifies operational risks, and provides a structured migration or modernization roadmap for enterprises running or evaluating Kafka.

    Kafka
    Read case study: Kafka Architecture Assessment and Migration Advisory
  • Assessment

    Apache Flink Streaming Readiness Assessment

    Smart Grid Technology ProviderEnergy

    Evaluating whether a proposed streaming workload belongs on Flink, and what the exactly-once, state, and operational requirements will really cost.

    Apache FlinkApache KafkaKubernetesRedpanda
    Read case study: Apache Flink Streaming Readiness Assessment
  • Remediation

    Debezium Oracle LogMiner Lag Remediation

    Regional Retail BankFinancial Services

    Bringing an Oracle change data capture pipeline back inside its batch window after nightly jobs left the Debezium connector hours behind the source.

    DebeziumOracleKafkaKubernetes
    Read case study: Debezium Oracle LogMiner Lag Remediation
  • Consulting

    Choosing Debezium + RabbitMQ over Kafka for Change Data Capture

    American National InsuranceHealthcare

    American National Insurance chose a standalone Debezium + RabbitMQ architecture for SQL Server change data capture after judging Kafka too complex to run for the job, keeping operations lean and routing flexible.

    KafkaRabbitMQDebeziumKubernetes
    Read case study: Choosing Debezium + RabbitMQ over Kafka for Change Data Capture
  • Support

    Debezium MySQL Binlog Retention and Schema History Support

    Online Travel PlatformSoftware

    Ongoing support for a MySQL change data capture pipeline that needed a full re-snapshot after a planned outage outlasted binlog retention and a trimmed schema history topic.

    DebeziumMySQLSQL ServerOracle DatabaseKafkaAmazon RDS
    Read case study: Debezium MySQL Binlog Retention and Schema History Support

Related Reading

Have a Kafka Challenge Like This?

AceMQ's senior Kafka engineers have handled this exact type of engagement before. Whether you need architectural guidance, hands-on remediation, or an ongoing managed partnership, we're ready to help.