Enum EvalTask 

pub enum EvalTask {
    NER {
        labels: Vec<String>,
        mode: EvalMode,
    },
    RelationExtraction {
        relations: Vec<String>,
        require_entity_match: bool,
    },
    Coreference {
        metrics: Vec<CorefMetric>,
    },
    DiscontinuousNER {
        labels: Vec<String>,
    },
    EventExtraction {
        event_types: Vec<String>,
        argument_roles: Vec<String>,
    },
}

Evaluation task type.

Clarifies what capability is being evaluated, since the same model may support multiple tasks with different metrics.

Example

use anno_eval::eval::{EvalTask, EvalMode};

let task = EvalTask::NER {
    labels: vec!["PER", "ORG", "LOC"].into_iter().map(String::from).collect(),
    mode: EvalMode::Strict,
};

match task {
    EvalTask::NER { labels, mode } => {
        println!("NER with {} entity types, {:?} mode", labels.len(), mode);
    }
    _ => {}
}

Variants

NER

Named Entity Recognition: span extraction with type classification.

Input: Text → Output: Vec<Entity>

Metrics: Precision, Recall, F1 (Strict/Exact/Partial/Type modes)

Fields

labels: Vec<String>

Entity type labels expected (e.g., [“PER”, “ORG”, “LOC”])

mode: EvalMode

Evaluation mode (Strict, Exact, Partial, Type)

RelationExtraction

Relation Extraction: entity pairs with relation types.

Input: Text → Output: Vec<(Entity, Relation, Entity)>

Metrics: Relation F1 (with/without entity correctness)

Fields

relations: Vec<String>

Relation types expected (e.g., [“WORKS_AT”, “BORN_IN”])

require_entity_match: bool

Whether to require correct entity spans for relation credit

Coreference

Coreference Resolution: mention clustering.

Input: Text → Output: Vec<CorefChain>

Metrics: MUC, B³, CEAF-e/m, LEA, BLANC, CoNLL F1

Fields

metrics: Vec<CorefMetric>

Which coreference metrics to compute

DiscontinuousNER

Discontinuous NER: non-contiguous span extraction.

Input: Text → Output: Vec<DiscontinuousEntity>

Metrics: Same as NER but with discontinuous span matching

Fields

labels: Vec<String>

Entity type labels expected

EventExtraction

Event Extraction: event triggers and arguments.

Input: Text → Output: Events with trigger and argument spans

Metrics: Trigger F1, Argument F1, Event F1

Fields

event_types: Vec<String>

Event types (e.g., [“ATTACK”, “MOVEMENT”, “TRANSACTION”])

argument_roles: Vec<String>

Argument roles (e.g., [“AGENT”, “PATIENT”, “LOCATION”])

Trait Implementations

impl Clone for EvalTask

fn clone(&self) -> EvalTask

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for EvalTask

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for EvalTask

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for EvalTask

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Eq for EvalTask

impl PartialEq for EvalTask

fn eq(&self, other: &EvalTask) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for EvalTask

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for EvalTask