Struct PairwiseJudge 

pub struct PairwiseJudge<M: BaseChatModel> { /* private fields */ }

Pairwise-comparison evaluator (an LLM as the judge, picks one of two).

P1-1: implements the PairwiseEvaluator trait and can join the pointwise Evaluators in EvalRunner unified report; calling compare directly still yields the fine-grained Verdict (A wins / B wins / tie).

Implementations

impl<M: BaseChatModel> PairwiseJudge<M>

pub fn new(judge: M) -> Self

Creates a pairwise-comparison evaluator using the default rubric.

pub fn with_rubric(self, rubric: impl Into<String>) -> Self

Sets a custom rubric (builder style).

pub async fn compare( &self, input: &str, a: &str, b: &str, ) -> Result<Verdict, EvalError>

Compares answers A and B, returning which is better.

Runs twice with A/B swapped to eliminate position bias: a consistent winner in both counts, otherwise a tie. P2-4: the two asks are independent and fire concurrently via future::join (eliminating the N+1 serial round-trips).

Trait Implementations

impl<M: BaseChatModel> PairwiseEvaluator for PairwiseJudge<M>

P1-1: enters EvalRunner as a PairwiseEvaluator, judging the two candidates as (a=prediction, b=reference). Score mapping: 1.0 = A wins, 0.5 = tie, 0.0 = B wins, and the label keeps the verdict meaning (a_wins / tie / b_wins).

fn eval_pair<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, input: &'life1 str, a: &'life2 str, b: &'life3 str, ) -> Pin<Box<dyn Future<Output = Result<Score, EvalError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Compares answers A and B, returning a 0-1 score (1.0 = A wins, 0.5 = tie, 0.0 = B wins).

fn name(&self) -> &str

Evaluator name (used in report summaries)