Expand description
Executor for Eval workflow nodes (3.12–3.17).
Runs configurable scorers against the preceding node’s output:
LlmJudge— calls an LLM with a rubric, extracts 1-5 scoreAssertion— evaluates Python-like boolean expressionsLatency— checks elapsed ms against a thresholdCost— checks estimated cost against a USD thresholdCustom— delegates to a Python scorer via module reference
On failure, the configured on_fail action determines the next step:
retry_with_feedback, escalate, halt, or log_and_continue.
Structs§
- Eval
Executor - Scorer
Result - Per-scorer result recorded in telemetry and execution output.