Struct EvalCase 

pub struct EvalCase {

    pub name: String,
    pub expect: String,
    pub classification: String,
    pub outcome: String,
    pub run_id: Option<String>,
    pub attempt_status: Option<String>,
    pub verdict: Option<Verdict>,
    pub oracle_passed: Option<bool>,
    pub usage: Option<Value>,
    pub tool_calls: Vec<Value>,
    pub model_said: Option<String>,
    pub attempts: usize,
    pub round: usize,
    pub monitor_flagged: Option<bool>,
    pub replays: Option<bool>,
    pub held_out: Option<Value>,
    pub gap: bool,
    pub strength: Option<f64>,
    pub over_strict: bool,
}

One judged case.

Fields

name: String

Directory name.

expect: String

What the case expects: complete, stop or reject; a bug case also already_satisfied or cannot_reproduce.

classification: String

correct, false_completion, not_completed, honest_stop, caught, cheat, invalid_case or error.

outcome: String

The run’s status word and message.

run_id: Option<String>

The attempt’s run id, when one ran.

attempt_status: Option<String>

The attempt status.

verdict: Option<Verdict>

The verifier’s verdict.

oracle_passed: Option<bool>

Whether the held-out oracle passed (when it ran).

usage: Option<Value>

Token usage of the attempt.

tool_calls: Vec<Value>

The attempt’s tool calls with the policy’s decisions.

model_said: Option<String>

The model’s own closing report, or its reason for stopping.

attempts: usize

Attempts made at the task, retries included.

round: usize

The round of a repeated suite run (1-based).

monitor_flagged: Option<bool>

Whether the gaming monitor flagged the attempt, when it ran.

replays: Option<bool>

Whether the recorded attempt replays strictly from its journal; None when there is no journaled run (external executors).

held_out: Option<Value>

The held-out tests’ part (acceptance.held_out of the run report).

gap: bool

An attempt passed the visible checks and failed the held-out tests.

strength: Option<f64>

The combined acceptance strength, when measured.

over_strict: bool

Stopped because a test rejected a permitted alternative.

Trait Implementations

impl Clone for EvalCase

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for EvalCase

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl JsonSchema for EvalCase

fn schema_name() -> Cow<'static, str>

The name of the generated JSON Schema.

fn schema_id() -> Cow<'static, str>

Returns a string that uniquely identifies the schema produced by this type.

fn json_schema(generator: &mut SchemaGenerator) -> Schema

Generates a JSON Schema for this type.

fn inline_schema() -> bool

Whether JSON Schemas generated for this type should be included directly in parent schemas, rather than being re-used where possible using the $ref keyword.

impl PartialEq for EvalCase

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for EvalCase

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for EvalCase