Enum DocumentScale 

pub enum DocumentScale {
    Short,
    Medium,
    Long,
    BookScale,
}

Document scale classification based on token count.

Research Context (Bourgois & Poibeau 2025, arXiv:2510.15594)

The paper shows that coreference performance degrades significantly with document length. These thresholds are informed by their analysis:

Scale           Token Range     Performance Impact
─────────────────────────────────────────────────────
Short           <2k             Baseline (OntoNotes-like)
Medium          2k-10k          -5% CoNLL F1
Long            10k-50k         -10% CoNLL F1
BookScale       >50k            -15% CoNLL F1, metrics unreliable

Example

use anno_eval::eval::DocumentScale;

let scale = DocumentScale::from_tokens(95_000);
assert!(scale.is_book_scale());
assert!(scale.metrics_may_be_unreliable());

Variants

Short

Short document (<2k tokens). OntoNotes-like scale.

Medium

Medium document (2k-10k tokens). Slight performance drop.

Long

Long document (10k-50k tokens). Noticeable degradation.

BookScale

Book-scale document (>50k tokens). Metrics may be unreliable.

Implementations

impl DocumentScale

pub fn from_tokens(token_count: usize) -> Self

Classify document scale from token count.

pub fn is_book_scale(&self) -> bool

Check if this is book-scale (>50k tokens).

pub fn metrics_may_be_unreliable(&self) -> bool

Check if coreference metrics may be unreliable at this scale.

At book scale, MUC tends to inflate while CEAF-e tends to collapse.

pub fn expected_degradation(&self) -> f64

Get expected CoNLL F1 degradation relative to short documents.

Trait Implementations

impl Clone for DocumentScale

fn clone(&self) -> DocumentScale

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for DocumentScale

impl Debug for DocumentScale

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for DocumentScale

fn default() -> DocumentScale

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for DocumentScale

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for DocumentScale

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for DocumentScale

impl PartialEq for DocumentScale

fn eq(&self, other: &DocumentScale) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for DocumentScale

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for DocumentScale