Struct EvalConfig 

pub struct EvalConfig {
    pub max_examples_per_dataset: usize,
    pub breakdown_by_difficulty: bool,
    pub breakdown_by_domain: bool,
    pub breakdown_by_type: bool,
    pub warmup: bool,
    pub warmup_iterations: usize,
    pub min_confidence: Option<f64>,
    pub cache_dir: Option<String>,
    pub normalize_types: bool,
}

Configuration for evaluation runs.

Fields

max_examples_per_dataset: usize

Maximum examples per dataset (0 = no limit)

breakdown_by_difficulty: bool

Include per-difficulty breakdown

breakdown_by_domain: bool

Include per-domain breakdown

breakdown_by_type: bool

Include per-entity-type metrics

warmup: bool

Run warmup iteration before timing

warmup_iterations: usize

Number of warmup iterations

min_confidence: Option<f64>

Minimum confidence threshold (filter predictions below this)

cache_dir: Option<String>

Cache directory for downloaded datasets

normalize_types: bool

Use type mapping to normalize domain-specific entity types to standard NER types.

When enabled:

• MIT Movie “Actor” -> “Person”
• MIT Restaurant “Restaurant_Name” -> “Organization”
• Biomedical “Disease” -> “Other” (or custom)

Implementations

impl EvalConfig

pub fn quick() -> Self

Quick evaluation (limited examples, no breakdowns).

pub fn full() -> Self

Full evaluation (all examples, all breakdowns).

pub fn ci_aware() -> Self

CI-aware configuration.

Respects environment variables:

• ANNO_MAX_EXAMPLES: Max examples per dataset (default: 50 in CI, 200 otherwise; set to 0 for unlimited)
• CI or GITHUB_ACTIONS: Detects CI environment

Example

# Limit to 20 examples per dataset
ANNO_MAX_EXAMPLES=20 cargo test -p anno-eval --features eval

pub fn with_type_normalization(self) -> Self

With type normalization enabled.

When enabled, domain-specific entity types are mapped to standard NER types:

• MIT Movie “Actor” -> “Person”
• MIT Restaurant “Restaurant_Name” -> “Organization”
• Biomedical “Disease” -> “Other”

Trait Implementations

impl Clone for EvalConfig

fn clone(&self) -> EvalConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for EvalConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for EvalConfig

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for EvalConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for EvalConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.