Struct ExperimentEngine 

pub struct ExperimentEngine { /* private fields */ }

Autonomous parameter-tuning engine.

The engine evaluates a baseline configuration, then generates and tests parameter variations one at a time. Accepted variations update the progressive baseline (greedy hill-climbing). The loop terminates on budget exhaustion, search-space exhaustion, wall-time limit, or cancellation.

Storage

When memory is Some, each result is persisted to SQLite via SemanticMemory::sqlite. When None, results are kept only in the in-memory results vec of the final report.

Budget ownership

The Evaluator is passed pre-built by the caller. The caller is responsible for constructing it with the desired budget_tokens (typically config.eval_budget_tokens). The eval_budget_tokens field in ExperimentConfig is a hint for the caller — the engine itself does not construct the evaluator.

Implementations

impl ExperimentEngine

pub fn new( evaluator: Evaluator, generator: Box<dyn VariationGenerator>, subject: Arc<AnyProvider>, baseline: ConfigSnapshot, config: ExperimentConfig, memory: Option<Arc<SemanticMemory>>, ) -> Self

Create a new ExperimentEngine.

A fresh UUID session ID is generated at construction time. The evaluator should already be configured with the desired token budget (typically config.eval_budget_tokens).

Contract

The caller must ensure config is valid before constructing the engine. Call ExperimentConfig::validate during bootstrap — passing invalid config (e.g., max_experiments=0, max_wall_time_secs=0) results in unspecified loop behaviour (immediate exit or no effective budget enforcement).

pub fn with_source(self, source: ExperimentSource) -> Self

Set the ExperimentSource for this session.

Defaults to ExperimentSource::Manual. Use ExperimentSource::Scheduled for runs triggered by the scheduler.

pub fn cancel_token(&self) -> CancellationToken

Return a clone of the internal CancellationToken.

External callers (CLI, TUI, scheduler) can hold a token handle and call .cancel() to trigger graceful shutdown. See also Self::stop.

pub fn stop(&self)

Stop the engine by cancelling the internal CancellationToken.

The current evaluation call will complete; the loop exits after it returns.

pub async fn run(&mut self) -> Result<ExperimentSessionReport, EvalError>

Run the experiment loop and return a session report.

The loop:

1. Evaluates the baseline once to obtain initial_baseline_score.
2. Generates variations via the VariationGenerator.
3. Evaluates each variation with a clone of subject patched with generation overrides derived from the candidate ConfigSnapshot via AnyProvider::with_generation_overrides.
4. Accepts the variation if delta >= config.min_improvement.
5. On acceptance, updates the progressive baseline (greedy hill-climbing). Known limitation (S1): single-sample acceptance has no statistical confidence check. Noise in the evaluator can cause gradual score drift. Phase 5 should add repeated trials or a confidence margin derived from per-case variance before promoting a variation.
6. Optionally persists results to SQLite when memory is Some.
7. Breaks on: max experiments, wall-time, search exhaustion, or cancellation.

Errors

Returns EvalError if the baseline evaluation or any subject LLM call fails. SQLite persistence failures are returned as EvalError::Storage.