anno-metrics

Shared evaluation and analysis primitives for anno: coreference scoring metrics and cluster encoders.

Note: as of 0.8.0 anno's facade has been collapsed (the anno-lib package was renamed to anno). Phase B will fold this crate's public types into anno::metrics::*. New code should plan to migrate.

Depends only on anno-core and serde, so it can be used by both anno (library) and anno-eval (evaluation harness) without creating dependency cycles.

Install

[dependencies]
anno-metrics = "0.6"

Coreference metrics

Implements the standard coreference scoring metrics used in CoNLL-2011/2012 and CRAC shared tasks:

Usage

use anno_core::{CorefChain, Mention};
use anno_metrics::coref_metrics::conll_f1;

let gold = vec![CorefChain::new(vec![
    Mention::new("John", 0, 4),
    Mention::new("he", 10, 12),
])];
let pred = gold.clone();

let f1 = conll_f1(&pred, &gold);
assert!((f1 - 1.0).abs() < 1e-9);

Output example

A runnable demo is at examples/scoring_demo.rs (cargo run -p anno-metrics --example scoring_demo). It constructs three gold entities, introduces a split error in one predicted cluster, and scores the result:

Metric     P       R       F1
------     -----   -----   -----
MUC        1.000   0.750   0.857
B3         1.000   0.810   0.895
CEAF-e     0.700   0.933   0.800
------
CoNLL F1:  0.851

Cluster encoder

Primitives for representing within-context clusters and scoring merges across contexts (document windows or separate documents), used in cross-document coreference pipelines. Key types:

License

Dual-licensed under MIT or Apache-2.0.