Azure Databricks

Required attributes

  • Workspace Url — the Databricks workspace URL (e.g., https://adb-1234567890123456.7.azuredatabricks.net).
  • Optional Default Notebook Path — workspace path scanned when importing standalone notebooks as jobs (e.g. /Shared). Leave blank to skip standalone-notebook discovery.

Compute configuration

Compute is read from the Platform's compute attributes when a run is submitted — see the Databricks Notebook page for how each is used: Compute Type (New Job Cluster, Existing Interactive Cluster, Existing Instance Pool, Serverless), Cluster Version, Cluster Node Type, Number of Worker Nodes, Existing Cluster ID and Existing Instance Pool. Leave them blank to reuse the job's own compute settings.

Authentication methods

The Authentication Method dropdown offers exactly two values, and the selected method is the one the provider uses — there is no silent fallback.

  • Service Principal ⭐ (recommended) — Tenant Id, Client Id, Client Secret. Polysync exchanges the app registration for an Entra ID token scoped to the Azure Databricks resource. Add the application as a service principal in the Databricks workspace (Admin Console → Identity and access) and grant it the required permissions. Rotate the secret regularly.
  • Access Token — Access Token. A personal access token (PAT) sent verbatim as the bearer token. Easy, but tied to a user account and rotated manually. Prefer the Service Principal for production.

Legacy records configured with any other value fail Test Connectivity with a clear "Unsupported authentication method" configuration error.

Permissions checklist

  • Add the chosen identity to the workspace and grant it Can Manage on the relevant Jobs (or Can Use on the SQL Warehouse / Cluster).
  • For Service Principal / Managed Identity auth, ensure the workspace has the principal added under Workspace > Admin Console > Identity and access > Service principals.

Supported jobs

Azure Databricks exposes four Polysync job types. See each dedicated page for parameter handling, output binding, execution flow, monitor URL, and troubleshooting:

Run outputs

Notebook, Job and Workflow runs read the dbutils.notebook.exit(...) value back after a successful run (via jobs/runs/get-output) when it is a JSON object: each top-level key that matches a parameter marked Output or Input & Output is written back into the run record. Plain-text exit values are not read. Delta Live Tables updates have no output mechanism.