Struct LLMEvaluator

pub struct LLMEvaluator { /* private fields */ }

Evaluator for comparing responses from multiple LLM providers

Implementations

impl LLMEvaluator

pub fn new(llms: Vec<Box<dyn LLMProvider>>) -> Self

Creates a new evaluator with the given LLM providers

Arguments

• llms - Vector of LLM providers to evaluate

pub fn scoring<F>(self, f: F) -> Self

where F: Fn(&str) -> f32 + Send + Sync + 'static,

Adds a scoring function to evaluate LLM responses

Arguments

• f - Function that takes a response string and returns a score

pub async fn evaluate_chat( &self, messages: &[ChatMessage], ) -> Result<Vec<EvalResult>, LLMError>

Evaluates chat responses from all providers for the given messages

Arguments

• messages - Chat messages to send to each provider

Returns

Vector of evaluation results containing responses and scores