Structs§
- Dataset
- A collection of input-reference pairs for evaluation.
- Dataset
Item - A single item in an evaluation dataset.
- Embedding
Distance Evaluator - Evaluator that computes cosine similarity between embeddings of prediction and reference.
- Eval
Case - Eval
Report - Eval
Result - Result of a single evaluation.
- Exact
Match Evaluator - Evaluator that checks for exact string match between prediction and reference.
- Json
Validity Evaluator - Evaluator that checks whether the prediction is valid JSON.
- LLMJudge
Evaluator - Evaluator that uses an LLM to judge prediction quality.
- Regex
Match Evaluator - Evaluator that checks whether the prediction matches a regex pattern.
Traits§
- Evaluator
- Trait for evaluating predictions against references.
Functions§
- evaluate
- Evaluate predictions against a dataset using an evaluator.