Struct RunConfig 

pub struct RunConfig {

    pub project: PathBuf,
    pub agent_dir: Option<PathBuf>,
    pub ir_path: Option<PathBuf>,
    pub golden_path: PathBuf,
    pub models: Vec<String>,
    pub out: Option<PathBuf>,
    pub parallel: usize,
    pub filter: CaseFilter,
    pub no_cache: bool,
    pub cache_dir: Option<PathBuf>,
    pub samples: usize,
    pub record_answers: bool,
    pub backend: Backend,
}

Fields

project: PathBuf

agent_dir: Option<PathBuf>

A dispatched agent output dir (contains .claude/agents/…) — the artifact Backend::ClaudeCodeCli installs and runs against. Required when backend is ClaudeCodeCli (the default); run_eval fails loudly if it’s missing then. Mutually exclusive with ir_path in practice — exactly one artifact shape applies per backend.

ir_path: Option<PathBuf>

Compiled IR JSON (the same artifact warble dispatch consumes) — the artifact Backend::ClaudeAgentSdk dispatches directly at invocation time, since that back-end has no pre-installed agent dir to point at. Required when backend needs it; run_eval fails loudly if it’s missing then.

golden_path: PathBuf

models: Vec<String>

out: Option<PathBuf>

parallel: usize

Concurrent cases per binding (1 = serial). Under contention the per-case latency column measures queueing too, so the report records the level it ran at.

filter: CaseFilter

Golden case selection (--tags / --sample). Default = full run (every case).

no_cache: bool

Bypass the trace cache (--no-cache): re-run every case and refresh its cached result instead of reusing a content-addressed hit.

cache_dir: Option<PathBuf>

Where per-case traces are read/written. None = <project>/.warble/eval-cache.

samples: usize

Repeated runs per case (pass-rate methodology). 1 is today’s single-run behavior; use more to distinguish model capability from run-to-run noise (flaky vs consistently passing/failing).

record_answers: bool

Record each sample’s rendered result value into CaseResult::answer_dist (a distinct- answer distribution). Off by default — it means keeping the actual answer strings around, not just pass/fail.

backend: Backend

Which back-end/runtime to dispatch the run through. Defaults to Backend::default (claude-code-cli) so an invocation that never mentions --backend measures exactly what it always measured.