Module measured 

Evaluation under rustev.evaluator-config/3 (spec 041, 3.2 and 3.4): ranking, short@k, backfill@p<p> and constraint metrics over the comparable proposal cases of a split, with their detail counts and the per-case measures spec 029 reconciles (3.11).

Every metric is computed over the split’s cases in manifest order. An incomparable, diverged or abstained case is outside every metric and counted by the reason the coverage counts use. A case a metric cannot measure is excluded and counted by reason, never scored 0 (I-2).

Modules

constraint_reason

Constraint exclusion reasons (3.4.6).

Structs

Measured

Version 3 evaluation inputs beside spec 004’s Inputs (3.4.1): the adapter that declares the measures whose digest the configuration binds.

NewMetric

One new metric: its report entry, its detail count, and each case’s measure (none outside its denominator), in manifest order.

Enums

Scope

One case of the split as the version 3 metrics see it.

Constants

SCORES_MISSING

Why a rubric metric has no value for its scorer (3.7.5).

Functions

agreement_metrics

The agreement metrics of 3.8.4 over cases scored on both sides: exact and adjacent (count ratios), mean_abs_diff (a mean) and qwk (not case-decomposable: no per-case measure). Deterministic agreements first, then those with a model side, named under model/. A side with no scores document makes every statistic unknown (scores-missing).

metrics

The version 3 deterministic metrics of 3.4, in declaration order: per ranking its metric specs, then short@k per declared k, then backfill@p<p>; then every constraint.

quadratic_weighted_kappa

Quadratic weighted kappa (3.8.4): 1 - sum(w * O) / sum(w * E), with w = (i - j)^2, E the outer product of the marginal counts divided by the case count, sums in row-major order; unknown when sum(w * E) = 0.

rubric_metrics

The rubric metrics of 3.6.3 over checked scores documents: the deterministic ones (rubric/<name>) and the model ones (model/rubric/<name>), each in declaration order, reported apart (3.2.4, I-3). A declared scorer with no document makes its metrics unknown (scores-missing); an unscored case is excluded and counted unscored:<reason>; a score outside the scorer’s output is unscored invalid_score, never clamped.