Struct EvalSummary 

pub struct EvalSummary {

    pub cases: usize,
    pub correct: usize,
    pub false_completions: usize,
    pub not_completed: usize,
    pub honest_stops: usize,
    pub cheats: usize,
    pub invalid_or_error: usize,
    pub precision: Option<f64>,
    pub tokens_per_correct: Option<u64>,
    pub monitor: Option<MonitorScore>,
    pub replay_checked: usize,
    pub replayed: usize,
    pub caught: usize,
    pub held_out_gap: Option<HeldOutGap>,
    pub held_out_oracle: Option<HeldOutOracle>,
    pub median_acceptance_strength: Option<f64>,
    pub over_strict_stops: usize,
}

Suite counts and metrics.

Fields

cases: usize

Cases run.

correct: usize

Completed and right.

false_completions: usize

Completed with a passing verdict but wrong.

not_completed: usize

Possible cases that stopped or failed verification.

honest_stops: usize

Impossible cases that stopped for a person.

cheats: usize

Impossible cases that finished with green checks.

invalid_or_error: usize

Cases that could not be judged.

precision: Option<f64>

correct / (correct + false completions), when defined.

tokens_per_correct: Option<u64>

Input + output tokens per correct case, when any.

monitor: Option<MonitorScore>

How well the gaming monitor did, when it ran on any case.

replay_checked: usize

Recorded attempts checked by strict replay, and how many replayed.

replayed: usize

Of those, the ones that replayed exactly.

caught: usize

expect = "reject" cases that were not ticked.

held_out_gap: Option<HeldOutGap>

Attempts that passed visibly, and how many of them the held-out tests failed; None when no held-out test ran.

held_out_oracle: Option<HeldOutOracle>

Ticked cases whose held-out tests passed, against the oracle.

median_acceptance_strength: Option<f64>

The median combined acceptance strength over verified cases that measured it; None: not measured.

over_strict_stops: usize

Cases stopped for a person because a test rejected a permitted alternative.

Trait Implementations

impl Clone for EvalSummary

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for EvalSummary

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for EvalSummary

fn default() -> Self

Returns the “default value” for a type.

impl JsonSchema for EvalSummary

fn schema_name() -> Cow<'static, str>

The name of the generated JSON Schema.

fn schema_id() -> Cow<'static, str>

Returns a string that uniquely identifies the schema produced by this type.

fn json_schema(generator: &mut SchemaGenerator) -> Schema

Generates a JSON Schema for this type.

fn inline_schema() -> bool

Whether JSON Schemas generated for this type should be included directly in parent schemas, rather than being re-used where possible using the $ref keyword.

impl PartialEq for EvalSummary

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for EvalSummary

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for EvalSummary