Databricks Workflow Job

The Databricks Workflow job type triggers a multi-task Databricks Job (a workflow with more than one task in its settings.tasks array) via POST jobs/run-now (Jobs API 2.1). The Databricks Job is identified by its numeric Job Id — Polysync stores that id (as a string) in the Job's External Id.

This job type is supported on the Azure Databricks platform.

Polysync's Databricks Job type covers single-task Jobs (any task kind); Databricks Workflow is the multi-task variant — discovery splits them automatically based on settings.tasks.length.

Required job fields

External Id — the numeric Databricks Job Id (e.g., 42). Parsed back to a long before the API call.
Job Type — Databricks Workflow (set automatically on import).

Job discovery

Polysync calls jobs/list (Jobs API 2.0) and then jobs/get for each job — the list payload omits settings.tasks, so classification happens on the detail response: when settings.tasks.length > 1, the Job is imported as a Workflow (otherwise as a Databricks Job). The detail call also populates the parameter schema from settings.job_parameters[], settings.parameters[], and any per-task base_parameters.

Parameter handling

Job-level parameters are passed as the job_parameters map on the run-now request:

{
  "job_id": <numeric-job-id>,
  "job_parameters": {
    "<parameter-name>": "<value>"
  }
}

Direction	Sent in `run-now`	Updated from response
`Input`	✅	❌
`Output`	❌	✅ (JSON exit object)
`Input&Output`	✅	✅ (JSON exit object)

Polysync infers the Data Type per parameter from each job_parameters[].default_value at import time (string / number / boolean), but Databricks ultimately receives all values as the parameter map shape it expects. Outputs are read back after a successful run: Polysync checks each task run's notebook exit value (jobs/runs/get-output) and maps the first JSON object found onto Output / Input & Output parameters. Tasks that are not notebooks produce no output.

Execution flow

Polysync posts to jobs/run-now with the resolved job_id and job_parameters.
The response run_id becomes the Polysync RunId.
Status is polled via GET jobs/runs/get?run_id={runId} using the same life_cycle_state + result_state decoding as Databricks Notebook.
When the run reaches result_state = SUCCESS, Polysync reads each task run's notebook exit value via jobs/runs/get-output and maps a JSON object's keys onto Output / Input & Output parameters.
Cancel is supported via POST jobs/runs/cancel.

Monitor URL

Polysync uses the run_page_url returned in the run-now response when present, otherwise falls back to {workspace_url}/#job/{run_id}/run/1.

Best practices

Define every parameter you want to override at the Job level (settings.job_parameters), not on individual tasks — task-level base_parameters are only seeded into the Polysync schema for reference, not forwarded at run time.
Pair this job type with Tasks in Polysync (not raw runs) so the dispatcher can manage retries, contention, and dependencies across Workflow runs.

Troubleshooting

INVALID_PARAMETER_VALUE: job_id — the External Id is not a valid numeric Job Id, or the Job was deleted. Re-import from the Platform editor.
Polysync parameter not honoured at run time — the parameter exists at the task level (base_parameters) but not at the job level (job_parameters). Edit the Databricks Job and declare the parameter on settings.job_parameters, then click Sync Parameters on the Polysync Job.

Documentation