Module eval_cases 

Per-case measures, rustev.eval-cases/1 (spec 029, 3.2).

One document per evaluation run: it binds the run’s report by record digest and lists, per case in manifest order, the case id, the run record it was measured from, its outcome key, its top-k list when declared, and its contribution to every case-decomposable metric. The measures reproduce the report’s figures bit for bit or the run is refused (reconcile), so a repeat set never aggregates numbers its own reports do not state.

Structs

CaseMeasures

EvalCases

Enums

CaseMeasure

One case’s contribution to one metric (3.2.2).

Constants

COST_PER_1000_ESTIMATED

COST_PER_1000_OBSERVED

The two metrics repeat reports derive (3.2.5); no run report states them, so reconciliation has nothing to compare them with.

EVAL_CASES

Functions

derived

Whether metric is one of the derived repeat-report metrics.

form_in

A metric’s form, by name or, for a spec 041 constraint or rubric metric, whose name does not say whether its aggregate is a mean (share_at_k, mean_level_index, mean) or a count ratio (all_at_k, share_at_least), by the kind of its first case measure (3.11.1).

form_of

How a spec 004 metric decomposes into per-case measures (3.2.3), by its reported name; none for a total (cost units, unknown attempts, liability), for brier (see below), or a metric no spec gives a per-case measure.

metric_cases

One metric’s contributions over a run set’s case documents: cases[case][run]. Every document must list the same case ids in the same order (3.1.3); otherwise the set is refused (mixed-runs), never truncated or padded.

reconcile

Whether one run’s case measures reproduce its report (3.2.6): the document binds the report’s record digest; its cases are the detail’s, in order, with the same outcome keys; every case-decomposable metric computed from the measures over the whole cohort equals the report’s value bit for bit (or both are unknown); and the count-ratio sums equal the detail’s counts.