Module evaluator

Evaluator for LLM providers Module for evaluating and comparing responses from multiple LLM providers.

This module provides functionality to run the same prompt through multiple LLMs and score their responses using custom evaluation functions.

Structs

EvalResult

Result of evaluating an LLM response

LLMEvaluator

Evaluator for comparing responses from multiple LLM providers

ParallelEvalResult

Result of a parallel evaluation including response, score, and timing information

ParallelEvaluator

Evaluator for running multiple LLM providers in parallel and selecting the best response

Type Aliases

ScoringFn

Type alias for scoring functions that evaluate LLM responses