Databricks Workflow Job

The Databricks Workflow job type triggers a multi-task Databricks Job (a workflow with more than one task in its settings.tasks array) via POST jobs/run-now (Jobs API 2.1). The Databricks Job is identified by its numeric Job Id — Polysync stores that id (as a string) in the Job's External Id.

This job type is supported on the Azure Databricks platform.

Polysync's Databricks Job type covers single-task / non-notebook Databricks Jobs. Databricks Workflow is the variant for multi-task Jobs — discovery splits them automatically based on settings.tasks.length.

Required job fields

External Id — the numeric Databricks Job Id (e.g., 42). Parsed back to a long before the API call.
Job Type — Databricks Workflow (set automatically on import).

Job discovery

Polysync calls jobs/list (Jobs API 2.0) and classifies each entry: when settings.tasks.length > 1, the Job is imported as a Workflow. A second call to jobs/get?job_id={id} populates the parameter schema from settings.job_parameters[], settings.parameters[], and any per-task base_parameters.

Parameter handling

Job-level parameters are passed as the job_parameters map on the run-now request:

{
  "job_id": <numeric-job-id>,
  "job_parameters": {
    "<parameter-name>": "<value>"
  }
}

Direction	Sent in `run-now`	Updated from response
`Input`	✅	❌
`Output`	❌	❌ (not supported)
`Input&Output`	✅	❌ (not supported)

Polysync infers the Data Type per parameter from each job_parameters[].default_value at import time (string / number / boolean), but Databricks ultimately receives all values as the parameter map shape it expects. Output parameters are not supported for this job type — multi-task workflows have no single return value.

Execution flow

Polysync posts to jobs/run-now with the resolved job_id and job_parameters.
The response run_id becomes the Polysync RunId.
Status is polled via GET jobs/runs/get?run_id={runId} using the same life_cycle_state + result_state decoding as Databricks Notebook.
Cancel is supported via POST jobs/runs/cancel.

Monitor URL

Polysync uses the run_page_url returned in the run-now response when present, otherwise falls back to {workspace_url}/#job/{run_id}/run/1.

Best practices

Define every parameter you want to override at the Job level (settings.job_parameters), not on individual tasks — task-level base_parameters are only seeded into the Polysync schema for reference, not forwarded at run time.
Pair this job type with Tasks in Polysync (not raw runs) so the dispatcher can manage retries, contention, and dependencies across Workflow runs.

Troubleshooting

INVALID_PARAMETER_VALUE: job_id — the External Id is not a valid numeric Job Id, or the Job was deleted. Re-import from the Platform editor.
Polysync parameter not honoured at run time — the parameter exists at the task level (base_parameters) but not at the job level (job_parameters). Edit the Databricks Job and declare the parameter on settings.job_parameters, then click Sync Parameters on the Polysync Job.

Documentation