Struct MultiTurnSimulation 

pub struct MultiTurnSimulation { /* private fields */ }

Multi-turn simulation for conversational/iterative model evaluation.

Implements Pareto frontier analysis across accuracy, cost, and latency.

Implementations

impl MultiTurnSimulation

pub fn new(seed: u64) -> Self

Create new multi-turn simulation.

pub fn with_costs(self, input_cost: f64, output_cost: f64) -> Self

Set cost parameters.

pub fn with_latency_per_token(self, latency_ms: f64) -> Self

Set latency per token.

pub fn turn<F>( &mut self, input: &str, expected: Option<&str>, generate_fn: F, ) -> SimResult<Turn>

where F: FnOnce(&str, &[Turn]) -> String,

Execute a single turn using a response generator.

The generate_fn takes (input, history) and returns response string.

Errors

Returns error if turn execution fails.

pub fn evaluate<F>( &mut self, queries: &[(String, Option<String>)], n_runs: usize, generate_fn: F, ) -> SimResult<MultiTurnEvaluation>

where F: Fn(&str, &[Turn]) -> String,

Run complete multi-turn evaluation with statistical analysis.

Following Princeton methodology: minimum 5 runs, 95% CI.

Errors

Returns error if fewer than 5 runs are requested or if evaluation fails.

pub fn pareto_analysis( evaluations: &[(String, MultiTurnEvaluation)], ) -> ParetoAnalysis

Compute Pareto frontier across multiple model evaluations.

pub fn history(&self) -> &[Turn]

Get conversation history.

pub fn reset(&mut self, seed: u64)

Reset simulation state.