Module evaluator

Source
Expand description

Evaluator for LLM providers Module for evaluating and comparing responses from multiple LLM providers.

This module provides functionality to run the same prompt through multiple LLMs and score their responses using custom evaluation functions.

Structs§

EvalResult
Result of evaluating an LLM response
LLMEvaluator
Evaluator for comparing responses from multiple LLM providers
ParallelEvalResult
Result of a parallel evaluation including response, score, and timing information
ParallelEvaluator
Evaluator for running multiple LLM providers in parallel and selecting the best response

Type Aliases§

ScoringFn
Type alias for scoring functions that evaluate LLM responses