pub struct Analyzer { /* private fields */ }Expand description
The text-analysis pipeline: normalize → tokenize → stop-word removal → optional stemming.
One Analyzer is configured per branch and used for both
indexing and querying, so an indexed term and the query term that
should match it pass through identical transformations. Build one
with Analyzer::builder; Analyzer::new gives the English
default (default stop words, no stemming).
Implementations§
Source§impl Analyzer
impl Analyzer
Sourcepub fn builder() -> AnalyzerBuilder
pub fn builder() -> AnalyzerBuilder
Start building a customized analyzer.
Sourcepub fn default_language(&self) -> Language
pub fn default_language(&self) -> Language
The language applied to text whose tag is absent or unrecognized.
Sourcepub fn analyze(&self, text: &str, lang_tag: Option<&str>) -> Vec<Term>
pub fn analyze(&self, text: &str, lang_tag: Option<&str>) -> Vec<Term>
Run the pipeline over text, producing its ordered terms.
lang_tag is the text’s optional BCP-47 language tag. When present
and recognized it selects the stemmer and stop-word list; otherwise
the analyzer’s default_language applies.
Duplicate terms are preserved (term frequency matters for scoring), in their original order.