Vertex AI Pipeline Job

The Vertex AI Pipeline Job job type submits a Kubeflow Pipelines / TFX Pipeline Job to Google Vertex AI via POST projects/{project}/locations/{location}/pipelineJobs. The template pipeline job is identified by its resource name — Polysync stores projects/{project}/locations/{location}/pipelineJobs/{jobId} in the Job's External Id.

This job type is supported on the Google Vertex AI platform.

Required job fields

External Id — the template Vertex AI Pipeline Job resource name.
Job Type — Vertex AI Pipeline Job (set automatically on import).
GCS Output Directory — captured as a job attribute from the template; the GCS path used as runtimeConfig.gcsOutputDirectory.

Job discovery

GET projects/{project}/locations/{location}/pipelineJobs?pageSize=100

(paginated). gcsOutputDirectory and the pipeline parameter list are captured.

Parameter handling

Input + Input&Output parameters are sent as runtimeConfig.parameterValues:

{
  "displayName": "<polysync-name>_<timestamp>",
  "runtimeConfig": {
    "gcsOutputDirectory": "<GCS Output Directory attribute>",
    "parameterValues": {
      "<param-1>": <typed-value>,
      "<param-2>": <typed-value>
    }
  }
}

Direction	Sent in `parameterValues`	Updated from response
`Input`	✅	❌
`Output`	❌	✅
`Input&Output`	✅	✅

Values are typed (string / int / bool / JSON / JSON array) and sent as JSON tokens.

Output parameters

On completion, Polysync reads runtimeConfig.parameterValues from the final pipeline state (Vertex AI echoes resolved values back) and populates Output / Input&Output parameters. If error.message is set, it's captured under the _error key.

Execution flow

Polysync POSTs the body above; the new resource name becomes the Polysync RunId.

Status is polled via GET projects/{project}/locations/{location}/pipelineJobs/{jobId} and decoded from state (PIPELINE_STATE_*):

Vertex AI `state`	Polysync status
`PIPELINE_STATE_SUCCEEDED`	Success
`PIPELINE_STATE_FAILED`	Failed
`PIPELINE_STATE_RUNNING` / `_QUEUED` / `_PENDING` / `_PAUSED`	Running
`PIPELINE_STATE_CANCELLING`	Running
`PIPELINE_STATE_CANCELLED`	Cancelled
(other)	Unknown

Cancel is supported via POST {resourceName}:cancel.

Monitor URL

https://console.cloud.google.com/vertex-ai/locations/{location}
  /pipelines/runs/{jobId}?project={projectId}

Best practices

Pre-compile the pipeline spec (pipeline.yaml / pipeline.json) and keep it under gcsOutputDirectory so the template Pipeline Job is reproducible.
Use Output parameters for IDs that downstream Tasks need (e.g., new model resource name) rather than scraping logs.

Troubleshooting

PERMISSION_DENIED on submit — the runtime identity lacks Vertex AI User and Storage Object Admin on gcsOutputDirectory.
Output parameter empty after success — the pipeline did not resolve a value for that parameter (it had no default and no Task produced it). Check the pipeline graph in the Monitor URL.

Documentation