Module ranking_metrics 

Ranking measures (spec 041, 3.4.2 to 3.4.4 and 3.11).

One case’s contribution to each ranking metric, from the ranked list a proposal carries and the case’s gold relevance. Unjudged items are removed from the top k (the condensed list) and counted, never read as non-relevant; a short list is not padded; a case a metric cannot measure is excluded with its reason, never scored 0 (I-2).

Values use integer arithmetic, binary64 addition, multiplication and division in the stated order, and the committed discount table, so they are bit-identical on every target (I-4). Nothing here reads a clock or performs I/O.

Modules

reason

Exclusion reasons of 3.3 and 3.4, as the detail breakdown names them.

Structs

Ranked

One entry of a ranked list: its id, and its origin when the list is a Selection of spec 039.

RankedList

A proposal’s ranked list, in position order (3.4.2).

RankingCase

One case’s result for one ranking metric.

Relevance

What a ranking declaration fixes for every case (3.4.2).

Enums

Gain

How a grade becomes a gain (3.4.3).

RankingMetric

A ranking metric specification (3.4.2): precision@k, recall@k, ndcg@k, hit@k, mrr or mrr@k, with k from 1 to 100.

Constants

MAX_K

The largest cutoff a ranking metric declares (3.4.2).

Functions

backfill_case

One case’s contribution to backfill@p<p> (3.4.4): whether the entry at position p came from backfill, over the lists that have a position p. A list without per-position origin is no_origin.

discount

The discount of condensed position i, 1 to 100.

ranked_list

The ranked list of a proposal parameter (3.4.2): a Ranking, a Shortlist or a Selection. Any other value, or none, is not_a_ranking; a list naming one id twice is invalid_ranking.

ranking_case

One case’s contribution to metric (3.4.3). gold is the case’s gold relevance map; none excludes the case (no_gold).

short_case

One case’s contribution to short@k (3.4.3): whether its list has fewer than k entries, over the cases with a ranking.