Struct AnnotationStore 

pub struct AnnotationStore;

Available on crate feature eval only.

Manages JSONL export and import for human annotation.

Implementations

impl AnnotationStore

pub fn export( cases: &[EvalCase], results: &[EvaluationResult], output_path: impl AsRef<Path>, ) -> Result<(), EvalError>

Export evaluation cases to a JSONL file for human review.

Each case is written as a single JSON line containing:

• case_id: the eval case identifier
• input: concatenated user input text from conversation turns
• expected_response: expected final response text (if available)
• actual_response: actual agent response from evaluation results (if available)
• verdict: always null on export (to be filled by human annotators)

Errors

Returns EvalError::AnnotationError if the file cannot be created or written to.

pub fn import( path: impl AsRef<Path>, valid_case_ids: &HashSet<String>, ) -> Result<(Vec<AnnotationRecord>, Vec<String>), EvalError>

Import annotations from a JSONL file.

Reads the file line-by-line, parsing each line as an AnnotationRecord. Validates that each record’s case_id exists in the provided set of valid IDs.

Returns

A tuple of:

• Valid annotation records (those with case_ids in the valid set)
• Warning messages for unmatched case_ids

Errors

Returns EvalError::AnnotationError if the file cannot be read or a line contains malformed JSON.