Documentation

Foundry Evaluation Run Job

The Foundry Evaluation Run job type starts an Azure AI Foundry Project evaluation run via POST /evaluations?api-version=v1. The target dataset is configured per Job, not per run; Polysync's External Id holds the placeholder name "evaluation" (evaluations are run instances, not pre-existing definitions).

This job type is supported on the Azure AI Foundry Project platform.

Required job fields

  • External Id — the placeholder "evaluation" (set automatically on import).
  • Job Type — Foundry Evaluation Run (set automatically on import).
  • Foundry Evaluation Target Dataset ID (job attribute) — required; the dataset to evaluate.
  • Foundry Evaluators (job attribute) — comma-separated names or a JSON object that becomes the request evaluators block.
  • Foundry Evaluation Name (job attribute) — optional display name; defaults to the job name.

Job discovery

Evaluations are not list-discoverable. A single placeholder Job is exposed per platform so users can schedule evaluations on demand.

Parameter handling

The request body is built from job attributes — Polysync parameters are surfaced as outputs only:

Polysync parameter Direction Role
evaluation_id Output Set to the new evaluation's id after submission.
status Output Set to the current status string on each poll.

Request body:

{
  "display_name": "<Foundry Evaluation Name>",
  "data": { "type": "dataset", "id": "<Foundry Evaluation Target Dataset ID>" },
  "evaluators": <parsed-from-Foundry-Evaluators>
}

The composite RunId is "eval#{evaluationId}".

Execution flow

  1. Polysync POSTs the body above; the response id is parsed into the RunId and the evaluation_id output parameter.

  2. Status is polled via GET /evaluations/{evaluationId}?api-version=v1:

    Foundry evaluation status Polysync status
    notstarted / queued / running Running
    completed / succeeded Success
    failed Failed

  3. Cancel is not supported — there is no documented cancel endpoint for Foundry evaluations.

Monitor URL

Same as Foundry Agent Run — generic Cognitive Services account overview.

Best practices

  • Treat the Polysync Evaluation Job as a trigger — design and curate the dataset and the evaluators in AI Foundry, then point Polysync at them via the job attributes.
  • Use Foundry Evaluators with a JSON object when you need to configure evaluator parameters (thresholds, model deployments, …).

Troubleshooting

  • HTTP 400 on submit — the Foundry Evaluation Target Dataset ID is missing or invalid. Check the dataset id in the Foundry portal.
  • Foundry Evaluators parse error — when not a comma-separated list, the value must be a valid JSON dictionary mapping evaluator names to their configs.