Skip to main content

Module eval

Module eval 

Source
Expand description

Executor for Eval workflow nodes (3.12–3.17).

Runs configurable scorers against the preceding node’s output:

  • LlmJudge — calls an LLM with a rubric, extracts 1-5 score
  • Assertion — evaluates Python-like boolean expressions
  • Latency — checks elapsed ms against a threshold
  • Cost — checks estimated cost against a USD threshold
  • Custom — delegates to a Python scorer via module reference

On failure, the configured on_fail action determines the next step: retry_with_feedback, escalate, halt, or log_and_continue.

Structs§

EvalExecutor
ScorerResult
Per-scorer result recorded in telemetry and execution output.