Module active_learning 

Active learning utilities for NER annotation.

Helps identify which examples to annotate next for maximum model improvement.

Sampling Strategies

• Uncertainty Sampling: Low-confidence predictions (requires confidence scores)
• Diversity Sampling: Examples most different from each other (requires embeddings)
• Query-by-Committee: High model disagreement (requires multiple model predictions)
• Hybrid: Combine uncertainty and committee signals

Strategy Requirements and Fallbacks

Each strategy has specific data requirements. When requirements aren’t met, the strategy falls back as follows:

Strategy	Requires	Falls back to
Uncertainty	confidence	Always works (uses 0.5 if missing)
Diversity	embedding	Uncertainty (with warning)
QueryByCommittee	committee_predictions (≥2)	Uncertainty (with warning)
Hybrid	Both confidence and committee	Uncertainty if committee missing

Example

use anno_eval::eval::active_learning::{ActiveLearner, SamplingStrategy, Candidate};

let learner = ActiveLearner::new(SamplingStrategy::Uncertainty);

let candidates = vec![
    Candidate::new("John works at Google.", 0.95),
    Candidate::new("Xiangjun joined Alibaba.", 0.45),  // Low confidence
];

let to_annotate = learner.select(&candidates, 1);
assert_eq!(to_annotate[0].text, "Xiangjun joined Alibaba.");

Structs

ActiveLearner

Active learning selector.

Candidate

A candidate example for annotation.

ScoreStats

Statistics about selection scores.

SelectionResult

Result of active learning selection.

Enums

SamplingStrategy

Sampling strategy for active learning.

Functions

entities_to_candidates

Convert extracted entities into active learning candidates.

estimate_budget

Estimate annotation budget needed for target performance.

export_annotation_priority

Export uncertainty-ranked entities as JSONL for annotation tools.

rank_for_annotation

Rank entities by annotation priority (lowest confidence first).

select_for_annotation

Export uncertainty-ranked entities as JSONL for annotation tools.