Struct EvalCodingAgentArgs 

pub struct EvalCodingAgentArgs {

    pub fixtures: Vec<String>,
    pub models: Vec<String>,
    pub tool_formats: Vec<String>,
    pub output: Option<PathBuf>,
    pub env_files: Vec<PathBuf>,
    pub include_local: bool,
    pub local_providers: Vec<String>,
    pub max_local_models: usize,
    pub keep_local_after_run: bool,
    pub max_runs: Option<usize>,
    pub max_iterations: usize,
    pub python: String,
    pub fail_on_unauthorized: bool,
    pub json: bool,
    pub step_judge: Option<String>,
    pub step_judge_on_veto: Option<String>,
    pub step_judge_adversarial: bool,
    pub override_reason: Option<String>,
    pub structural_validator: Option<String>,
    pub run_label: String,
    pub baseline_comparison_against: Option<PathBuf>,
}

Fields

fixtures: Vec<String>

Fixture ids to run (comma-separated, repeatable). Use all for the full suite.

models: Vec<String>

Model selectors to run (comma-separated, repeatable). Each entry may be an alias, provider:model, or provider=...,model=....

tool_formats: Vec<String>

Tool-call rendering modes to compare.

output: Option<PathBuf>

Output directory for summary.json, per_run.jsonl, transcripts, and markdown reports.

env_files: Vec<PathBuf>

Optional .env file(s) to load for provider credentials. Values are never written to artifacts.

include_local: bool

Append reachable local Ollama/llama.cpp/MLX/vLLM models to the selected matrix.

local_providers: Vec<String>

Restrict local discovery to one provider id. Repeatable.

max_local_models: usize

Maximum discovered local models to append.

keep_local_after_run: bool

Leave newly-loaded Ollama models running after each local benchmark run.

max_runs: Option<usize>

Stop after N matrix entries, useful for cost-capped smoke runs.

max_iterations: usize

Maximum repair-agent loop iterations per run.

python: String

Python executable used by the fixture and verification command.

fail_on_unauthorized: bool

Treat missing credentials as an error instead of skipping the run.

json: bool

Print the aggregate summary JSON to stdout.

step_judge: Option<String>

Optional step_judge config applied to every run in this invocation. Accepts a preset name (symmetric-cheap, asymmetric, symmetric-strong) which expands to a known {model, provider} pair, or custom:<json> for a literal JSON object passed verbatim to agent_loop({step_judge: ...}). Omit (or pass none / off) to disable. For matrix sweeps across presets, the step-judge experiment driver at experiments/step-judge/run.sh invokes the eval runner once per preset and aggregates.

step_judge_on_veto: Option<String>

Override the on_veto remediation shape for the step-judge config (replace or retain). Default is replace.

step_judge_adversarial: bool

Use the adversarial rubric variant.

override_reason: Option<String>

Free-form reason attached when forcing a tool format against catalog guidance.

structural_validator: Option<String>

Structural-validator config applied to every run in this invocation. Omit to use the suite default (currently the 4-rule validator). Accepts on / default, off / none, or custom:<json> for a literal JSON object passed to with_structural_validator(...).

run_label: String

Free-form label persisted in summary.json for grouping repeat runs (e.g. “replicate-1”, “probe-judge-arch-gpt”). Defaults to empty.

baseline_comparison_against: Option<PathBuf>

Path to a previous coding-agent summary.json (or its parent dir). When present, the new summary embeds a baseline_comparison block listing per-fixture regressions (baseline passed but this cell failed) and recoveries (baseline failed but this cell passed), plus aggregate counts and a net lift in percentage points. Useful for cross-cell A/Bs (provider sweep, prompt change, step judge on/off) where net pass-rate hides destructive interactions like the cli-help-flag regression the step-judge experiment surfaced (harn#2318).

Trait Implementations

impl Args for EvalCodingAgentArgs

fn group_id() -> Option<Id>

Report the ArgGroup::id for this set of arguments

fn augment_args<'b>(__clap_app: Command) -> Command

Append to Command so it can instantiate Self via FromArgMatches::from_arg_matches_mut

fn augment_args_for_update<'b>(__clap_app: Command) -> Command

Append to Command so it can instantiate self via FromArgMatches::update_from_arg_matches_mut

impl Debug for EvalCodingAgentArgs

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl FromArgMatches for EvalCodingAgentArgs

fn from_arg_matches(__clap_arg_matches: &ArgMatches) -> Result<Self, Error>

Instantiate Self from ArgMatches, parsing the arguments as needed.

fn from_arg_matches_mut( __clap_arg_matches: &mut ArgMatches, ) -> Result<Self, Error>

Instantiate Self from ArgMatches, parsing the arguments as needed.

fn update_from_arg_matches( &mut self, __clap_arg_matches: &ArgMatches, ) -> Result<(), Error>

Assign values from ArgMatches to self.

fn update_from_arg_matches_mut( &mut self, __clap_arg_matches: &mut ArgMatches, ) -> Result<(), Error>

Assign values from ArgMatches to self.