Struct RunConfig 

pub struct RunConfig {

    pub project: PathBuf,
    pub agent_dir: Option<PathBuf>,
    pub ir_path: Option<PathBuf>,
    pub golden_path: PathBuf,
    pub models: Vec<String>,
    pub out: Option<PathBuf>,
    pub parallel: usize,
    pub filter: CaseFilter,
    pub no_cache: bool,
    pub cache_dir: Option<PathBuf>,
    pub samples: usize,
    pub record_answers: bool,
    pub backend: Backend,
    pub max_turns: Option<u32>,
    pub tier_models: Option<ModelConfig>,
}

Fields

project: PathBuf

agent_dir: Option<PathBuf>

A dispatched agent output dir (contains .claude/agents/…) — the artifact Backend::ClaudeCodeCli installs and runs against. Required when backend is ClaudeCodeCli (the default); run_eval fails loudly if it’s missing then. Mutually exclusive with ir_path in practice — exactly one artifact shape applies per backend.

ir_path: Option<PathBuf>

Compiled IR JSON (the same artifact warble dispatch consumes) — the artifact Backend::ClaudeAgentSdk dispatches directly at invocation time, since that back-end has no pre-installed agent dir to point at. Required when backend needs it; run_eval fails loudly if it’s missing then.

golden_path: PathBuf

models: Vec<String>

out: Option<PathBuf>

parallel: usize

Concurrent cases per binding (1 = serial). Under contention the per-case latency column measures queueing too, so the report records the level it ran at.

filter: CaseFilter

Golden case selection (--tags / --sample). Default = full run (every case).

no_cache: bool

Bypass the trace cache (--no-cache): re-run every case and refresh its cached result instead of reusing a content-addressed hit.

cache_dir: Option<PathBuf>

Where per-case traces are read/written. None = <project>/.warble/eval-cache.

samples: usize

Repeated runs per case (pass-rate methodology). 1 is today’s single-run behavior; use more to distinguish model capability from run-to-run noise (flaky vs consistently passing/failing).

record_answers: bool

Record each sample’s rendered result value into CaseResult::answer_dist (a distinct- answer distribution). Off by default — it means keeping the actual answer strings around, not just pass/fail.

backend: Backend

Which back-end/runtime to dispatch the run through. Defaults to Backend::default (claude-code-cli) so an invocation that never mentions --backend measures exactly what it always measured.

max_turns: Option<u32>

The --max-turns cap to invoke the back-end’s own dispatch CLI with, or None to leave its default turn budget in place. Only Backend::ClaudeAgentSdk has a --max-turns knob at all (see ClaudeAgentSdkAdapter::invoke); run_eval fails loudly, before any spend, if this is Some for any other backend rather than silently ignoring it — see validate_max_turns_backend. Also part of the trace cache key (CaseKey::max_turns) so a capped run can never hit a differently-capped (or uncapped) run’s cached trace.

tier_models: Option<ModelConfig>

A differentiated --strong/--cheap/--orchestrator (or --models-config) binding for this run, or None to keep the pre-existing single-model --models sweep (models above) exactly as it always behaved. Some bypasses the models sweep entirely and runs one pass with this resolved binding instead — see run_eval. Only Backend::ClaudeAgentSdk can honour a differentiated binding; run_eval fails loudly before any spend otherwise (see validate_tier_binding_backend).