Struct instant_segment::Segmenter

pub struct Segmenter { /* fields omitted */ }

Central data structure used to calculate word probabilities

Implementations

impl Segmenter

pub fn from_iters<U, B>(unigrams: U, bigrams: B) -> Self where
    U: Iterator<Item = (String, f64)>,
    B: Iterator<Item = ((String, String), f64)>, 

Create Segmenter from the given iterators

Note: the String types used in this API are defined in the smartstring crate. Any &str or String can be converted into the String used here by calling into() on it.

pub fn from_maps(
    unigrams: HashMap<String, f64, RandomState>,
    bigrams: HashMap<(String, String), f64, RandomState>
) -> Self

Create Segmenter from the given hashmaps (using ahash)

Note: the String types used in this API are defined in the smartstring crate. Any &str or String can be converted into the String used here by calling into() on it. The HashMap type here refers to std::collections::HashMap parametrized with the ahash::RandomState.

pub fn segment<'a>(
    &self,
    input: &str,
    search: &'a mut Search
) -> Result<impl Iterator<Item = &'a str> + ExactSizeIterator, InvalidCharacter>

Segment the text in input

Requires that the input text consists of lowercase ASCII characters only. Otherwise, returns Err(InvalidCharacter). The search parameter contains caches that are used segmentation; passing it in allows the callers to reuse the cache allocations.

pub fn set_limit(&mut self, limit: usize)

Customize the word length limit

pub fn set_total(&mut self, total: f64)

Customize the relative score by setting the total