Struct TestGenerator 

pub struct TestGenerator { /* private fields */ }

Available on crate feature eval only.

Generates evaluation test cases from descriptions or event logs.

Implementations

impl TestGenerator

pub fn new(model: Arc<dyn Llm>) -> TestGenerator

Creates a new test generator with default configuration.

pub fn with_config( model: Arc<dyn Llm>, config: GeneratorConfig, ) -> TestGenerator

Creates a new test generator with custom configuration.

pub async fn generate_from_description( &self, description: &str, ) -> Result<Vec<EvalCase>, EvalError>

Generate eval cases from a natural language description.

Prompts the LLM to produce eval case definitions as JSON. On parse failure for any individual case, a warning is logged and that case is skipped without aborting the batch.

pub fn generate_from_events( &self, events: &[Event], ) -> Result<Vec<EvalCase>, EvalError>

Generate eval cases from production event logs.

Extracts conversation turns from the provided events, constructing EvalCase objects directly without invoking the LLM.