Struct Analyzer 

pub struct Analyzer { /* private fields */ }

A complete text analysis pipeline: char filters + tokenizer + token filters.

Combines zero or more CharFilters, a single Tokenizer, and zero or more TokenFilters. Character filters preprocess the raw text, the tokenizer breaks it into tokens, then each token filter transforms the stream in order.

See [[analyzers#Pipeline Stages]].

Implementations

impl Analyzer

pub fn new( name: impl Into<String>, tokenizer: impl Tokenizer + 'static, filters: Vec<Box<dyn TokenFilter>>, ) -> Self

Create a new analyzer with the given name, tokenizer, and filters.

pub fn with_char_filters( name: impl Into<String>, char_filters: Vec<Box<dyn CharFilter>>, tokenizer: impl Tokenizer + 'static, filters: Vec<Box<dyn TokenFilter>>, ) -> Self

Create a new analyzer with char filters, tokenizer, and token filters.

pub fn from_boxed( name: impl Into<String>, char_filters: Vec<Box<dyn CharFilter>>, tokenizer: Box<dyn Tokenizer>, filters: Vec<Box<dyn TokenFilter>>, ) -> Self

Create from already-boxed components (used by config builder).

pub fn analyze(&self, text: &str) -> Vec<Token>

Run the full analysis pipeline on the input text.

Applies char filters, tokenizes, corrects offsets, then applies token filters in order.

pub fn name(&self) -> &str

The analyzer’s name (e.g., "standard", "simple").