Skip to main content

Crate snowstem

Crate snowstem 

Source
Expand description

Snowball stemming for 21 languages.

Each algorithm module is generated from the current Snowball release and checked against the official test vocabularies word for word. The crate has no required dependencies, builds without std, and stems a word with at most one allocation. Serde support for Algorithm is an opt-in serde feature.

use snowstem::{Algorithm, Stemmer};

let stemmer = Stemmer::create(Algorithm::English);
assert_eq!(stemmer.stem("international"), "internat");
assert_eq!(stemmer.stem("internal"), "internal");

Input is expected in lower case. Snowball defines stemming over lowercase words, so casefold and tokenize before calling.

§Reusing one buffer

Stemmer::stem_into writes the stem into a String you own, so a loop over many words allocates only when the buffer has to grow:

use snowstem::{Algorithm, Stemmer};

let stemmer = Stemmer::create(Algorithm::German);
let mut stem = String::new();
for word in ["häuser", "gelaufen"] {
    stemmer.stem_into(word, &mut stem);
    assert!(!stem.is_empty());
}

§Dutch

Snowball 3.0 replaced its Dutch algorithm. Algorithm::Dutch is the current one. Algorithm::DutchPorter is the earlier algorithm, kept for indexes built with it, since the two produce different stems for about half of the Dutch test vocabulary.

Structs§

Stemmer
A stemmer for one Algorithm.
UnknownAlgorithm
The error from parsing a name no Algorithm carries.

Enums§

Algorithm
A stemming algorithm, named after the language it handles.