Struct ParallelEvaluator

pub struct ParallelEvaluator { /* private fields */ }

Evaluator for running multiple LLM providers in parallel and selecting the best response

Implementations

impl ParallelEvaluator

pub fn new(providers: Vec<(String, Box<dyn LLMProvider>)>) -> Self

Creates a new parallel evaluator

Arguments

• providers - Vector of (id, provider) tuples to evaluate

pub fn scoring<F>(self, f: F) -> Self

where F: Fn(&str) -> f32 + Send + Sync + 'static,

Adds a scoring function to evaluate LLM responses

Arguments

• f - Function that takes a response string and returns a score

pub fn include_timing(self, include: bool) -> Self

Sets whether to include timing information in results

pub async fn evaluate_chat_parallel( &self, messages: &[ChatMessage], ) -> Result<Vec<ParallelEvalResult>, LLMError>

Evaluates chat responses from all providers in parallel for the given messages

Arguments

• messages - Chat messages to send to each provider

Returns

Vector of evaluation results containing responses, scores, and timing information

pub async fn evaluate_chat_with_tools_parallel( &self, messages: &[ChatMessage], tools: Option<&[Tool]>, ) -> Result<Vec<ParallelEvalResult>, LLMError>

Evaluates chat responses with tools from all providers in parallel

Arguments

• messages - Chat messages to send to each provider
• tools - Optional tools to use in the chat

Returns

Vector of evaluation results

pub async fn evaluate_completion_parallel( &self, request: &CompletionRequest, ) -> Result<Vec<ParallelEvalResult>, LLMError>

Evaluates completion responses from all providers in parallel

Arguments

• request - Completion request to send to each provider

Returns

Vector of evaluation results

pub fn best_response<'a>( &self, results: &'a [ParallelEvalResult], ) -> Option<&'a ParallelEvalResult>

Returns the best response based on scoring

Arguments

• results - Vector of evaluation results

Returns

The best result or None if no results are available