Orchestrating the Azure data platform
Most Azure data estates are not one service. A typical overnight run loads data with Azure Data Factory, transforms it in Databricks or a Fabric notebook, and refreshes a Power BI semantic model at the end. Each of those services can schedule its own work, but none of them owns the whole chain. Orchestrating the Azure data platform means making that chain explicit: what runs first, what waits for what, and what happens when a step fails.
What each service orchestrates on its own
| Service | Native orchestration | Where it stops |
|---|---|---|
| Azure Data Factory | Pipelines with activity dependencies, schedule, tumbling-window and event triggers | Dependencies resolve inside a pipeline; work started elsewhere is not a node in the graph |
| Microsoft Fabric Data Factory | Pipelines with control flow, schedules and event triggers, inside a Fabric workspace | Work outside Fabric is reached through activities or connectors, not as a peer job |
| Databricks Lakeflow Jobs | Multi-task jobs (a DAG) with time and file-arrival triggers | Tasks run on Databricks; upstream work in other services is outside the graph |
| Azure Logic Apps | Trigger-and-action workflows with 1,400+ connectors | Designed for integration and business processes rather than data-pipeline DAGs |
| Power BI | Scheduled refresh per semantic model | A schedule is a clock time, not "after the load finished" |
Sources: Microsoft Learn (Fabric Data Factory overview, Logic Apps overview) and Databricks (Lakeflow Jobs), checked 7 November 2026.
Common patterns, and their trade-offs
- Nest everything in one ADF or Fabric pipeline. Call Databricks and Power BI from pipeline activities. Simple when one team owns everything; harder when each team owns its own service and its own schedule.
- Time gaps. Schedule each service at a time that "should" be after the previous one. Cheap, and wrong whenever a run is late.
- Event glue. Storage events, webhooks or a queue. Reliable when built well, but it is code to own.
- A control plane over the services. Each job stays where it is; dependencies, parameters and schedules live in one place. This is the approach Polysync takes.
How Polysync covers the Azure services
Polysync connects to Azure Data Factory, Azure Synapse pipelines, Microsoft Fabric (notebooks, Spark job definitions, pipelines and dataflows), Azure Databricks (notebooks, single-task jobs, multi-task workflows and pipelines), Azure Functions, Logic Apps, Power BI (semantic model and dataflow refresh), Azure Batch, AKS, Azure Automation, SQL databases, Azure OpenAI and Azure AI Foundry. It discovers the jobs in each, imports their parameters, and lets you link them with On Success, On Failure or On Completion dependencies.
Outputs travel where the service exposes them: an ADF pipeline's pipelineReturnValue, a Databricks notebook's JSON exit value, or a stateful Logic Apps workflow's outputs can be mapped into the next task's inputs.
Frequently asked questions
Is Microsoft Fabric replacing Azure Data Factory?
Microsoft describes Data Factory in Fabric as the next generation of Azure Data Factory and publishes an upgrade guide, but Azure Data Factory remains a separate, supported service. Many estates run both during a migration, which is a common reason to orchestrate across them.
Can Power BI refresh automatically when my pipeline finishes?
Not with a scheduled refresh alone, because a schedule is a clock time. You need something to call the refresh API after the load succeeds: a pipeline activity, a script, or an orchestrator dependency.
Do I need to rebuild my ADF pipelines to use an orchestrator?
No. Polysync imports existing pipelines and calls them as they are. The pipeline logic stays in ADF.
Comparisons in this topic
- Polysync vs Microsoft Fabric Data FactoryOrchestration inside Microsoft Fabric versus orchestration across Fabric and everything around it.
- Polysync vs Databricks Lakeflow JobsOrchestration inside a Databricks workspace versus Databricks as one platform in a wider graph.
- Azure Data Factory alternatives, and when to keep ADFFabric, Airflow, Lakeflow Jobs, Logic Apps and control planes compared with ADF, including the option to keep it.
- Polysync vs Azure Logic Apps for data orchestrationEvent-driven integration workflows versus scheduled data-pipeline orchestration, and how they combine.
Answers in this topic
- How do I trigger a Databricks job after an Azure Data Factory pipeline finishes?ADF's Databricks activity, a Jobs API call, a file-arrival trigger or an orchestrator dependency, compared.
- How do I refresh a Power BI semantic model only after my ETL succeeds?Replace a clock-time scheduled refresh with a refresh that waits for the load to succeed.
- How do I orchestrate Microsoft Fabric pipelines alongside existing Azure Data Factory pipelines?Keeping one overnight chain working while pipelines move from ADF to Fabric.
- Do I need Apache Airflow if I already use Azure Data Factory?Add Airflow for code-first workflows; not just to sequence pipelines ADF already runs.