Back to all use cases

AssessmentCross-Industry

A readiness label and an owner for every pipeline before the first one moves

Data Platform Team

A typical assessment for a data team with Azure Data Factory and Synapse pipelines that must move to Fabric: readiness per pipeline, the items that need rebuilding by hand, an order of moves and a rollback for the first wave.

Microsoft Fabric consulting

Overview

This is a typical engagement for a data team with a few hundred Azure Data Factory pipelines, some Synapse pipelines and a dedicated SQL pool, who want to work in Fabric. Nobody has counted which pipelines will upgrade cleanly. AceMQ engineers, with a partner subject-matter expert where needed, run the assessment and write the plan.

Challenge

Microsoft's built-in upgrade experience labels each pipeline and activity Ready, Needs review, Coming soon or Not compatible. Datasets become inline properties, global parameters become variable libraries, self-hosted integration runtimes are replaced by on-premises data gateways, Key Vault integration is more limited, and connection properties do not support dynamic values, so parameterized linked services need redesign. Some activities, such as Validation, Notebook, Jar, Python and the five HDInsight activities, have no direct equivalent and are rebuilt with other activities.

Environment

Azure Data Factory factories with parameterized linked services, global parameters, self-hosted integration runtimes, Mapping Data Flows and a metadata-driven pipeline framework, plus Synapse pipelines and one dedicated SQL pool.

Approach

AceMQ engineers mount a factory in a Fabric workspace as an Azure Data Factory item, run the pipeline assessment, and inventory pipelines, linked services, integration runtimes, triggers and data flows. They group pipelines by label and by business impact, and design the replacement for each Not compatible item. The first wave is rehearsed in a test workspace.

Solution

  1. Inventory of pipelines, linked services, integration runtimes, triggers and data flows with owners
  2. Assessment results per pipeline and the reason behind each Needs review or Not compatible label
  3. Replacement design for unsupported activities and for parameterized connections
  4. Plan for global parameters as variable libraries and for runtime and gateway replacement
  5. Order of moves by business impact, with the first wave small enough to prove the route
  6. Cutover per domain with the old pipeline kept as the rollback

Outcome

The engagement delivers a written assessment with a label and an owner for every pipeline, the replacement designs, the order of moves and a rehearsal plan for the first wave. The customer's team runs the upgrade from the plan, or AceMQ can run it alongside them.

Technologies

Microsoft FabricAzure Data FactoryAzure Synapse

Ready for a Microsoft Fabric Health Check?

AceMQ's senior Microsoft Fabric engineers have handled this exact type of engagement before. Whether you need architectural guidance, hands-on remediation, or an ongoing managed partnership, we're ready to help.