Crate anno 

anno

Information extraction for unstructured text: named entity recognition (NER), coreference resolution, relation extraction, PII detection, and zero-shot entity types.

• NER output: variable-length spans with character offsets (Unicode scalar values), not byte offsets.
• Coreference output: clusters (“tracks”) of mentions within one document.
• Relation output: (head, relation, tail) triples via RelationExtractor backends.
• PII detection: pii module for detecting and redacting personally identifiable information.
• RAG preprocessing: rag::preprocess chunks text, extracts entities, and rewrites pronouns for self-contained retrieval chunks.
• Export: export module for brat, CoNLL, JSONL, N-Triples, JSON-LD, and graph CSV.

This crate focuses on inference-time extraction. Dataset loaders, benchmarking, and matrix evaluation tooling live in anno-eval (and the anno CLI lives in anno-cli).

Quickstart

use anno::{Model, StackedNER};

let m = StackedNER::default();
let ents = m.extract_entities("Lynn Conway worked at IBM and Xerox PARC.", None)?;
for e in &ents {
    println!("{} [{}] ({},{}) {:.2}", e.text, e.entity_type, e.start(), e.end(), e.confidence);
}
// Lynn Conway [PER] (0,12) 0.95
// IBM [ORG] (27,30) 0.95
// Xerox PARC [ORG] (35,45) 0.95

Zero-shot custom entity types

Zero-shot custom entity types are provided by GLiNER backends when the onnx feature is enabled. See the repo docs for the CLI flag (--extract-types) and the library API.

Offline / downloads

By default, ML weights may download on first use. Set ANNO_NO_DOWNLOADS=1 to block new HuggingFace fetches; cached models and backends loaded from local paths (via from_local or the ONNX export scripts) still work. The flag is checked at the HF-download boundary, not at backend construction, so local-only pipelines are unaffected.

Threading

Extraction is CPU-bound and synchronous. Backends are Send + Sync and thread-safe for concurrent extract_entities calls on a shared reference. In async services, wrap per-document extraction in tokio::task::spawn_blocking (or use rayon’s par_iter for batch work on a single model).

Re-exports

pub use error::Error;

pub use error::Result;

pub use crate::core::generate_span_candidates;

pub use crate::core::Animacy;

pub use crate::core::Confidence;

pub use crate::core::CorefChain;

pub use crate::core::CorefDocument;

pub use crate::core::CoreferenceResolver;

pub use crate::core::Corpus;

pub use crate::core::DiscontinuousSpan;

pub use crate::core::Entity;

pub use crate::core::EntityBuilder;

pub use crate::core::EntityCategory;

pub use crate::core::EntityType;

pub use crate::core::ExtractionMethod;

pub use crate::core::Gender;

pub use crate::core::GroundedDocument;

pub use crate::core::HashMapLexicon;

pub use crate::core::HierarchicalConfidence;

pub use crate::core::Identity;

pub use crate::core::IdentityId;

pub use crate::core::IdentitySource;

pub use crate::core::Lexicon;

pub use crate::core::Location;

pub use crate::core::Mention;

pub use crate::core::MentionType;

pub use crate::core::Modality;

pub use crate::core::Number;

pub use crate::core::Person;

pub use crate::core::PhiFeatures;

pub use crate::core::Provenance;

pub use crate::core::Quantifier;

pub use crate::core::RaggedBatch;

pub use crate::core::Relation;

pub use crate::core::Signal;

pub use crate::core::SignalId;

pub use crate::core::SignalRef;

pub use crate::core::Span;

pub use crate::core::SpanCandidate;

pub use crate::core::Track;

pub use crate::core::TrackId;

pub use crate::core::TrackRef;

pub use crate::core::TrackStats;

pub use crate::core::TypeLabel;

pub use crate::core::TypeMapper;

pub use crate::core::ValidationIssue;

pub use crate::core::grounded::SignalValidationError;

pub use crate::core::types::ByteOffset;

pub use crate::core::types::CanonicalId;

pub use crate::core::types::CharOffset;

pub use lang::detect_language;

pub use lang::Language;

pub use offset::bytes_to_chars;

pub use offset::chars_to_bytes;

pub use offset::is_ascii;

pub use offset::OffsetMapping;

pub use offset::SpanConverter;

pub use offset::TextSpan;

pub use offset::TokenSpan;

pub use similarity::string_similarity;

pub use types::EntitySliceExt;

pub use backends::ConflictStrategy;

pub use backends::CrfNER;

pub use backends::EnsembleNER;

pub use backends::HeuristicNER;

pub use backends::LexiconNER;

pub use backends::NuNER;

pub use backends::RegexNER;

pub use backends::StackedNER;

pub use backends::TPLinker;

pub use backends::W2NERConfig;

pub use backends::W2NERRelation;

pub use backends::W2NER;

pub use backends::coref::mention_ranking::ClusteringStrategy;

pub use backends::coref::mention_ranking::MentionCluster;

pub use backends::coref::mention_ranking::MentionRankingConfig;

pub use backends::coref::mention_ranking::MentionRankingCoref;

pub use backends::coref::mention_ranking::RankedMention;

pub use backends::CorefBackend;

pub use backends::inference::extract_relation_triples;

pub use backends::inference::extract_relation_triples_simple;

pub use backends::inference::extract_relations;

pub use backends::inference::CoreferenceConfig;

pub use backends::inference::DiscontinuousEntity;

pub use backends::inference::DiscontinuousNER;

pub use backends::inference::ExtractionWithRelations;

pub use backends::inference::RelationExtractionConfig;

pub use backends::inference::RelationExtractor;

pub use backends::inference::RelationTriple;

pub use backends::inference::ZeroShotNER;

pub use backends::BertNEROnnx;onnx

pub use backends::GLiNEROnnx;onnx

pub use backends::FCoref;onnx

pub use backends::FCorefConfig;onnx

pub use backends::CandleNER;candle

pub use backends::stacked::StackedExtractionError;

pub use backends::stacked::StackedExtractionPolicy;

pub use backends::stacked::StackedExtractionReport;

pub use backends::stacked::StackedLayerOutcome;

Modules

active

Active learning utilities for annotation prioritization.

backends

NER backend implementations.

coalesce

Coalescing primitives shared across coreference and cross-doc identity resolution.

core

Core types (Entity, Span, Track, Confidence, …) and their submodules.

discoursediscourse

Discourse-level analysis: centering theory, abstract anaphora, dialogue acts.

edit_distance

Edit distance algorithms. Edit distance utilities for fuzzy string matching.

env

Environment variable utilities.

error

Error types for anno.

export

Export entity results to annotation and interchange formats (brat, CoNLL, JSONL, RDF, JSON-LD, CSV). Export entity extraction results to annotation and interchange formats.

graphgraph

Graph / knowledge-graph export adapters (lattix-backed).

heuristics

Small, dependency-light heuristics (negation, quantifiers, etc.). Small, dependency-light heuristics shared across the repo.

ingest

Lightweight URL/file ingestion helpers (not a crawling/pipeline product). Document ingestion and text preparation.

lang

Language detection and classification utilities.

metricsanalysis

Coreference scoring metrics (MUC, B³, CEAF, LEA, BLANC, CoNLL F1) and cluster-encoding primitives.

minimal

Lite re-export facade for crates that only need data types (no algorithms). Lightweight import surface for downstream crates.

models

Default model identifiers for backend construction.

offset

Unified byte/character/token offset handling.

pii

PII detection and redaction (library-level privacy functions). PII (personally identifiable information) detection and redaction.

prelude

Common imports for working with anno.

rag

Coreference preprocessing for RAG: rewrite pronouns for self-contained chunks.

schema

Schema harmonization for multi-dataset NER training.

similarity

Text similarity for entity matching and coreference resolution.

types

Type-level programming patterns for compile-time safety.

Structs

AnnotatedDoc

Text paired with its extraction outputs (entities, relations, coreference chains).

AnyModel

A wrapper that turns an extractor closure into a Model.

ModelCapabilities

Runtime discovery mechanism for model capabilities behind Box<dyn Model>.

OnnxSessionConfigonnx and (candle or onnx)

Configuration for creating an ONNX Runtime session.

Traits

Model

Trait for NER model backends.

Functions

annotate

Extract entities from text using the default backend and return an AnnotatedDoc.

annotate_grounded

Extract a document with the default backend into the canonical grounded representation.

annotate_grounded_batch_with

Extract multiple identified documents with model into grounded documents.

annotate_grounded_with

Extract one document with model into a GroundedDocument.

auto

Automatically select the best available NER backend.

available_backends

List backends and whether their required features are compiled into this build.

create_onnx_sessiononnx and (candle or onnx)

Create an ONNX Runtime session from a model file with the given configuration.

download_model_fileonnx and (candle or onnx)

Download a file from a HuggingFace repo, trying multiple candidate paths in order.

extract

Extract entities from text using the best available backend.

extract_batch

Extract entities from multiple texts using the best available backend.