Databricks Job

The Databricks Job job type triggers a single-task Databricks Job (notebook, Python script, JAR, SQL — any task kind) via POST jobs/run-now (Jobs API 2.1). The Databricks Job is identified by its numeric Job Id — Polysync stores that id (as a string) in the Job's External Id.

This job type is supported on the Azure Databricks platform.

Multi-task Databricks Jobs are imported as Databricks Workflow instead. Standalone notebooks are imported as Databricks Notebook.

Required job fields

External Id — the numeric Databricks Job Id (e.g., 42). Parsed back to a long before the API call.
Job Type — Databricks Job (set automatically on import).

Job discovery

Polysync lists jobs with jobs/list (Jobs API 2.0), then classifies each from its jobs/get detail — the list payload omits settings.tasks, so the task count is only visible on the detail response:

Exactly one task (any kind — notebook, script, SQL, …) → Databricks Job.
More than one task → Databricks Workflow.
Standalone notebooks are discovered from the workspace tree (workspace/list) as Databricks Notebook — a job record is never imported as that type.

The same jobs/get response populates the parameter schema from settings.job_parameters[], settings.parameters[], and the task's base_parameters.

Parameter handling

Identical to Databricks Workflow — values are sent through job_parameters on the run-now request:

{
  "job_id": <numeric-job-id>,
  "job_parameters": {
    "<parameter-name>": "<value>"
  }
}

Direction	Sent in `run-now`	Updated from response
`Input`	✅	❌
`Output`	❌	✅ (JSON exit object)
`Input&Output`	✅	✅ (JSON exit object)

Outputs are read back after a successful run from the task's notebook exit value (jobs/runs/get-output) when it is a JSON object whose keys match Output / Input & Output parameters. A non-notebook task has no output mechanism.

Execution flow

Polysync posts to jobs/run-now.
The response run_id becomes the Polysync RunId.
Status is polled via GET jobs/runs/get?run_id={runId} using the same life_cycle_state + result_state mapping as the other Databricks job types (see table).
When the run reaches result_state = SUCCESS, Polysync reads the notebook exit value via jobs/runs/get-output and fills any Output / Input & Output parameters from a JSON-object exit value.
Cancel is supported via POST jobs/runs/cancel.

Monitor URL

Polysync uses run_page_url from the run-now response when present, otherwise {workspace_url}/#job/{run_id}/run/1.

Troubleshooting

Job not found on import — jobs/list paginates; very large workspaces may exceed the default page size. Re-run Get Pipelines to fetch the next page, or filter from the Databricks UI to confirm the Job exists.
Parameter ignored at run time — declare the parameter on the Databricks Job's job-level parameters rather than the task's base_parameters, then click Sync Parameters.

Documentation