Expand description
Scorer, rubric, human scale, agreement and origin declarations of
rustev.evaluator-config/3 (spec 041, 3.5 to 3.8).
These are declared data: a scorer’s descriptor names its identity,
output and rules, so it enters the EvaluatorConfigId. A model scorer
is a judge whose plan and backend identities are bound here too. The
checks refuse every declaration that would mix a model metric with a
deterministic one, read an ordinal scale as the wrong kind, or report an
origin family before its evidence is read.
Modules§
- code
- Refusal codes of spec 041 under
invalid_input(3.10.5).
Structs§
- Agreement
Decl - An agreement between two ordinal sides (3.8.4): each a declared
scorer id or
human:<scale>. - Declarations
- The version 3 members of an evaluator configuration (3.1.3).
- Human
Scale - A human rating scale (3.6.4).
- Judge
Ref - A judge plan and the backend identities serving it (3.8.1).
- Rubric
Metric - A rubric metric (3.6.3), reported as
rubric/<name>or, for a model scorer,model/rubric/<name>. - Scorer
Descriptor - A scorer’s descriptor (3.6.2).
Enums§
- Origin
Family - An origin and evidence-quality family (3.5.1).
- Rubric
Aggregate - Scorer
Kind - Scorer
Output - What a score is (3.6.2).
Constants§
- HUMAN_
SIDE - An agreement side naming a human scale (3.8.4).
- MAX_
LEVELS - An ordinal scale holds 2 to 16 levels (3.6.2).
- MAX_
SCORERS - The most scorers one configuration declares (3.6.2).
- MODEL_
PREFIX - The reserved prefix of every model-derived metric (3.8.5).
Functions§
- check_
prefix - The model-prefix rule for a metric a gate or formula names (3.8.5): a
rubric or agreement metric of a model scorer is named under
model/, and a deterministic one never is.