Struct Judge 

pub struct Judge { /* private fields */ }

A model that grades language.

It is a separate provider from the one under test on purpose: an agent that grades its own prose is measuring its own preferences.

Implementations

impl Judge

pub fn new(provider: Arc<dyn ModelProvider>) -> Self

A judge backed by provider.

pub fn with_system_prompt(self, system: impl Into<String>) -> Self

Replaces the system prompt. The replacement still only ever sees a question and an answer.

pub const fn with_temperature(self, temperature: Option<f32>) -> Self

Sets the sampling temperature. Zero by default, because a judge that wanders is a judge whose votes measure nothing.

pub fn verdict_schema() -> Value

The JSON Schema a verdict must satisfy.

pub fn prompt(criterion: JudgeCriterion, input: &JudgeInput) -> String

The user message one vote carries.

Pure and public so a test can assert what a judge is shown — and, more usefully, what it is not shown.

pub async fn vote( &self, criterion: JudgeCriterion, input: &JudgeInput, ) -> Result<JudgeVerdict, JudgeError>

Collects one opinion.

Errors

• JudgeError::Provider when the model call failed;
• JudgeError::Schema when the verdict schema could not be compiled;
• JudgeError::Malformed when the answer was not a verdict;
• JudgeError::OutOfRange when the score is outside MIN_SCORE..=MAX_SCORE.

pub async fn poll( &self, criterion: JudgeCriterion, input: &JudgeInput, votes: u32, ) -> CriterionOutcome

Collects votes opinions about one sample and returns them with their aggregate.

Votes never fail the run: a vote that could not be obtained is recorded as a vote without a verdict, and the majority is taken over the rest. A judge that is down is a fact about the judge, not a regression in the agent under test.

Trait Implementations

impl Debug for Judge

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.