Orchestrating the Azure data platform

Most Azure data estates are not one service. A typical overnight run loads data with Azure Data Factory, transforms it in Databricks or a Fabric notebook, and refreshes a Power BI semantic model at the end. Each of those services can schedule its own work, but none of them owns the whole chain. Orchestrating the Azure data platform means making that chain explicit: what runs first, what waits for what, and what happens when a step fails.

What each service orchestrates on its own

Service Native orchestration Where it stops
Azure Data Factory Pipelines with activity dependencies, schedule, tumbling-window and event triggers Dependencies resolve inside a pipeline; work started elsewhere is not a node in the graph
Microsoft Fabric Data Factory Pipelines with control flow, schedules and event triggers, inside a Fabric workspace Work outside Fabric is reached through activities or connectors, not as a peer job
Databricks Lakeflow Jobs Multi-task jobs (a DAG) with time and file-arrival triggers Tasks run on Databricks; upstream work in other services is outside the graph
Azure Logic Apps Trigger-and-action workflows with 1,400+ connectors Designed for integration and business processes rather than data-pipeline DAGs
Power BI Scheduled refresh per semantic model A schedule is a clock time, not "after the load finished"

Sources: Microsoft Learn (Fabric Data Factory overview, Logic Apps overview) and Databricks (Lakeflow Jobs), checked 7 November 2026.

Common patterns, and their trade-offs

  1. Nest everything in one ADF or Fabric pipeline. Call Databricks and Power BI from pipeline activities. Simple when one team owns everything; harder when each team owns its own service and its own schedule.
  2. Time gaps. Schedule each service at a time that "should" be after the previous one. Cheap, and wrong whenever a run is late.
  3. Event glue. Storage events, webhooks or a queue. Reliable when built well, but it is code to own.
  4. A control plane over the services. Each job stays where it is; dependencies, parameters and schedules live in one place. This is the approach Polysync takes.

How Polysync covers the Azure services

Polysync connects to Azure Data Factory, Azure Synapse pipelines, Microsoft Fabric (notebooks, Spark job definitions, pipelines and dataflows), Azure Databricks (notebooks, single-task jobs, multi-task workflows and pipelines), Azure Functions, Logic Apps, Power BI (semantic model and dataflow refresh), Azure Batch, AKS, Azure Automation, SQL databases, Azure OpenAI and Azure AI Foundry. It discovers the jobs in each, imports their parameters, and lets you link them with On Success, On Failure or On Completion dependencies.

Outputs travel where the service exposes them: an ADF pipeline's pipelineReturnValue, a Databricks notebook's JSON exit value, or a stateful Logic Apps workflow's outputs can be mapped into the next task's inputs.

Frequently asked questions

Is Microsoft Fabric replacing Azure Data Factory?

Microsoft describes Data Factory in Fabric as the next generation of Azure Data Factory and publishes an upgrade guide, but Azure Data Factory remains a separate, supported service. Many estates run both during a migration, which is a common reason to orchestrate across them.

Can Power BI refresh automatically when my pipeline finishes?

Not with a scheduled refresh alone, because a schedule is a clock time. You need something to call the refresh API after the load succeeds: a pipeline activity, a script, or an orchestrator dependency.

Do I need to rebuild my ADF pipelines to use an orchestrator?

No. Polysync imports existing pipelines and calls them as they are. The pipeline logic stays in ADF.

Comparisons in this topic

Answers in this topic

Data pipeline orchestration for Azure-first teams

Keep your ADF, Databricks, Fabric and Power BI jobs. Polysync connects them with shared scheduling, cross-platform dependencies and one run history, including jobs on AWS and Google Cloud. No orchestration code to write.

Learn more about Polysync