Struct EvalReport 

pub struct EvalReport {
    pub suite: String,
    pub generated_at: DateTime<Utc>,
    pub config: EvalConfig,
    pub items: Vec<ItemReport>,
}

One whole run.

Fields

suite: String

The suite that ran.

generated_at: DateTime<Utc>

When the run finished.

config: EvalConfig

The configuration it ran under, so a reader can see how many samples and how many votes produced these numbers.

items: Vec<ItemReport>

Every item, in suite order.

Implementations

impl EvalReport

pub fn new( suite: impl Into<String>, generated_at: DateTime<Utc>, config: EvalConfig, items: Vec<ItemReport>, ) -> Self

Builds a report.

pub fn item(&self, id: &ItemId) -> Option<&ItemReport>

One item by identifier.

pub fn total_samples(&self) -> usize

Total samples across the run.

pub fn samples_passed(&self) -> usize

Samples that satisfied every deterministic expectation.

pub fn deterministic_pass_rate(&self) -> f64

Fraction of samples that satisfied every deterministic expectation.

pub fn flaky_items(&self) -> Vec<&ItemReport>

Items whose samples disagreed with each other.

pub fn reliability(&self) -> Reliability

The dashboard of §26.3.

pub fn judge_summaries(&self) -> Vec<CriterionSummary>

The judge’s numbers across the whole suite.

pub fn to_json(&self) -> Result<String, ReportError>

The machine-readable form, for a continuous integration gate to archive and for crate::baseline to read back.

Errors

ReportError::Serialize when the report cannot be rendered as JSON.

pub fn from_json(source: &str) -> Result<Self, ReportError>

Reads a report back from its machine-readable form.

Errors

ReportError::Deserialize when the document is not a report.

pub fn summary(&self) -> String

A readable summary, one block per item plus the dashboard.

pub fn task_accuracy(&self) -> Option<String>

Each understanding task’s accuracy over the samples whose item says what it expects of that task, as segment 40/46 · route 30/31; None when no item says.

pub fn discard_codes(&self) -> String

The discarded-answer codes of the whole run, commonest first, rendered as code×n.

Which code it is decides what to do about it, and the two that this corpus produces want opposite answers: evidence means the model cited something the turn does not contain, which is a reading the grounding rule caught, while schema_violation means the document was the wrong shape, which is a schema the model cannot satisfy. Reporting only a rate would leave the reader unable to tell them apart.

pub fn gate(&self, thresholds: &GateThresholds) -> GateOutcome

Applies a gate’s thresholds.

Trait Implementations

impl Clone for EvalReport

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for EvalReport

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for EvalReport

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for EvalReport

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for EvalReport

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for EvalReport