Struct StopwordSet 

pub struct StopwordSet { /* private fields */ }

A sorted set of stopwords supporting O(log n) lookup.

Construct once per process with StopwordSet::builtin and reuse across segmentation calls.

Implementations

impl StopwordSet

pub fn builtin() -> Self

Load the built-in Thai stopword list (1 029 entries, PyThaiNLP Apache-2.0).

pub fn from_text(data: &str) -> Self

Build a StopwordSet from a newline-separated word list.

Lines beginning with # and blank lines are ignored. BOM characters (\u{FEFF}) are stripped from every line. The resulting set is sorted and deduplicated.

pub fn builtin_with_extra(extra: &str) -> Self

Load the built-in list plus additional words from extra.

extra uses the same format as from_text: newline-separated words, # comment lines and blank lines ignored, BOM stripped. The combined set is sorted and deduplicated.

Use this when you have domain-specific function words to suppress in addition to the standard Thai stopword list.

Example

use kham_core::stopwords::StopwordSet;

let stops = StopwordSet::builtin_with_extra("ดาวน์โหลด\nอัปโหลด\n");
assert!(stops.contains("และ"));       // built-in
assert!(stops.contains("ดาวน์โหลด")); // extra

pub fn contains(&self, word: &str) -> bool

Return true if word is in the stopword set.

pub fn len(&self) -> usize

Number of stopwords in this set.

pub fn is_empty(&self) -> bool

Return true if the set is empty.