Expand description
Metric functions — numbers computed from matches.
Every item here is a free function over flat arrays, callable without constructing an evaluator:
use hotcoco::metrics::counts::average_precision;
let scores = [0.9, 0.8, 0.3];
let matched = [true, false, true];
let ap = average_precision(&scores, &matched, None, 3, &[0.0, 0.5, 1.0]);This is the same shape sklearn.metrics and torchmetrics.functional use,
and for the same reason: a metric is a pure function of its inputs, so tying
it to an evaluator object only makes it harder to reach.
§metrics vs primitives
The split is by what a function produces, not by which family calls it:
| Produces | Contains | |
|---|---|---|
primitives | matches and similarities | sim, greedy, assign |
metrics | numbers from matches | counts, calibration, confusion, bootstrap |
primitives::greedy::greedy_match_masked decides which detection pairs
with which ground truth. metrics::counts::average_precision turns that
decision into a number. Nothing here matches; nothing there scores.
tests/architecture.rs enforces the direction of the dependency: metrics
may not import from a family driver such as detection,
and primitives may not import from metrics.
Taking (scores, matched) rather than a family-specific struct is what makes
these reusable across families: detection produces those arrays from
eval_imgs, tracking and panoptic will produce them from their own match
records, and the function does not know which called it.
§Degenerate-input convention
These free functions share one policy, the same as primitives:
- Mismatched parallel-array lengths are a programmer error and panic via
assert!with a message naming both lengths. Nothing silently truncates, no-ops, or degrades (primitives::assign::lsapset the pattern). Each function’s# Panicssection states its checks. - Empty input is not an error — it produces the documented empty-set
value (
0.0, an emptyVec, an all-zero matrix), because “no detections” is a legitimate evaluation state, not a bug.
§Stability
Provisional through 1.x, like primitives: public so the
family drivers and Python can share them, but not frozen until 1.4. Expect
additive change — new functions, and the tracking count vocabulary in
counts — rather than reshaping of what is here.
Modules§
- bootstrap
- Bootstrap confidence intervals over any resampled statistic.
- calibration
- Confidence calibration — reliability bins, ECE, and MCE.
- confusion
- Confusion counts over matched prediction/ground-truth pairs.
- counts
- Count aggregation and metric formulas.
Functions§
- is_
computed - Whether a metric value was actually computed, as opposed to carrying the
crate’s
-1.0“not computed for this configuration” sentinel. - is_
missing - The complement of
is_computed:vis the “not computed” sentinel.