language-text-analysis 0.1.3

Small, deterministic multilingual text-analysis pipeline for search and document processing
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12

//! Built-in default stop-word lists.
//!
//! Each list is stored **already normalized** — lowercase and
//! accent-stripped, matching the form a token has after the
//! [`normalize`](crate) stage — so stop-word removal can compare
//! tokens directly without re-normalizing the list. The lists are
//! deliberately compact common-word sets, not exhaustive linguistic
//! corpora; callers wanting maximum recall select
//! [`StopWordPolicy::None`](crate::StopWordPolicy).

pub(crate) mod english;
pub(crate) mod german;