Module report 

Evaluating a dataset split into a report (spec 004, 3.5).

Every case of the split is reproduced from its bundle; with a candidate, each reproduced case is also compared. The existing rustev.eval-report/1 envelope carries the metrics; a rustev.eval-detail/1 document binds the report’s digest, dataset, split and configuration to every numerator, denominator and exclusion and to each case’s scope, outcome and reason. A standalone envelope never proves those denominators. All counts reconcile to the named cohort before a report is returned.

Structs

Candidate

A candidate plan with the calibration artifacts it binds.

CandidateBinding

The candidate’s binding and the model it serves.

CaseDetail

ClaimRequest

What a claim asks to state. Its scope is not free text: dataset, split, configuration and binding come from the freeze, and the population must be the admitted one.

Count

One metric’s accounting: numerator of denominator, and every case the metric left out or broke down, by reason.

EvalDetail

Evaluated

A report with the documents that explain it.

FreezeInputs

What a freeze takes.

Frozen

A checked, digested freeze. Only freeze makes one.

Inputs

What one evaluation takes.

Measurement

The side-by-side measurement of one frozen qualification.

QualificationRecord

What a qualification fixes before the final test (spec 023, 3.3.1). Its digest is the frozen identity. It holds no binary64 number.

RoleFigures

One role’s figures: coverage and abstention rows always come before accuracy and probability rows (spec 023, R-1).

Row

ScopedClaim

A claim sentence and the limits that bound it (spec 023, 3.5).

SliceRows

One slice’s figures.

Enums

ClaimError

Why a claim was rejected.

EvalError

Why no report was produced.

ExactBaseline

The exact baseline: the task’s exact plan steps, or why the task allows none (spec 023, 3.4.6).

Figure

One figure. A ratio carries its numerator and denominator; a value is a spec 004 measurement with its count. Unknown carries the reason a figure cannot be given (for example a zero denominator, a slice below its minimum, or exposed final-test cases); it is never replaced by a number, and scope_claim refuses to compose a claim from it.

GateResult

A gate’s verdict. Unknown is never a pass.

QualificationError

Why a qualification step refused.

Role

A report’s role in a qualification.

Served

The served model identity, as observed for the calls a claim covers (spec 012, 3.6.3).

Constants

EVAL_DETAIL

QUALIFICATION

Schema of a qualification record’s frozen part (spec 023, 3.3).

Functions

evaluate

Evaluate one split of a dataset.

evaluate_gate

A gate over a baseline and a candidate evaluation. The cohorts compared are the ones the reports name; they must share dataset, split and configuration, and each detail must be bound to its report.

evaluate_measured

Evaluate one split under a version 3 configuration (spec 041), with the adapter that declares its measures. Under versions 1 and 2 this is evaluate, byte for byte; plain evaluate refuses a version 3 configuration, which binds measures it has no declarations for.

evaluate_measured_with_cases

evaluate_with_cases under a version 3 configuration (spec 041, 3.11): the case measures carry every new metric’s contributions.

evaluate_views

The score views of a version 3 evaluation (spec 041, 3.7.1): the same function evaluate_measured uses, so the same inputs give the same views and record digest. Scores in measured are checked as evaluation checks them; pass none to compute views to score.

evaluate_with_cases

Evaluate one split and write its per-case measures (spec 029, 3.2): the same report, detail and configuration as evaluate, byte for byte, with a rustev.eval-cases/1 document bound to the report and already reconciled with it.

freeze

Freeze a qualification before any final-test case is evaluated (spec 023, 3.3). Refuses unadmitted data, a configuration other than the frozen one or with unbound rules, a candidate whose family labeled the data, a calibration without lineage or touching the final test, slices without minimums, and a replacement that reuses the old final test.

measure

Measure a frozen qualification from spec 004 reports of its roles on the final-test split (spec 023, 3.4). Every report must be of the admitted dataset, the final-test split, every final-test case and the frozen configuration and bindings; a report under another configuration is QualificationError::ConfigChanged. Exposed cases are counted apart and keep every accuracy and probability figure unknown, since a spec 004 report cannot leave them out. A slice below its frozen minimum, and a headline below the sample-size target, is unknown, never a figure.

regression

Regression against the previous qualified result for the same task (spec 023, 3.4.5): each frozen gate over the previous result’s report (baseline role) and this one’s (candidate role), by spec 004’s own evaluate_gate, which is unknown unless both share dataset, split and configuration. Empty when the freeze names no previous result; unknown when it names one that is not supplied.

scope_claim

Compose a scoped claim (spec 023, 3.5, I-3). The sentence names the dataset id, split, evaluator configuration id, binding identity, call dates, served identity disclosure and population; the limits carry the admission’s stated limits. Anything beyond them is rejected.

split_name