snowstem 0.1.0

Snowball stemming for 21 languages, generated from the current Snowball release, with no required dependencies and no_std support
Documentation
  • Coverage
  • 100%
    28 out of 28 items documented1 out of 7 items with examples
  • Size
  • Source code size: 814.8 kB This is the summed size of all the files inside the crates.io package for this release.
  • Documentation size: 2.4 MB This is the summed size of all files generated by rustdoc for all configured targets
  • Ø build duration
  • this release: 4s Average build duration of successful builds.
  • all releases: 4s Average build duration of successful builds in releases after 2024-10-23.
  • Links
  • hey-jj/snowstem
    0 0 0
  • crates.io
  • Dependencies
  • Versions
  • Owners
  • hey-jj

snowstem

Snowball stemming for 21 languages.

Each algorithm module is produced by the official Snowball compiler, version 3.1.1, and checked word for word against the Snowball project's test vocabularies. The crate has no required dependencies, builds without std, and stems a word with at most one allocation. The minimum supported Rust version is 1.63.

Usage

use snowstem::{Algorithm, Stemmer};

let stemmer = Stemmer::create(Algorithm::English);
assert_eq!(stemmer.stem("international"), "internat");
assert_eq!(stemmer.stem("internal"), "internal");

Input is expected in lower case. Snowball defines stemming over lowercase words, so casefold and tokenize before calling. Stemmer::stem_into writes the stem into a String you own, so a loop over many words stops allocating once the buffer fits the longest word.

The opt-in serde feature adds Serialize and Deserialize for Algorithm as its lowercase name. The default build depends on no other crate.

Languages

Arabic, Czech, Danish, Dutch, English, Finnish, French, German, Greek, Hungarian, Italian, Norwegian, Polish, Portuguese, Romanian, Russian, Spanish, Swedish, Tamil, and Turkish.

Snowball 3.0 replaced its Dutch algorithm. Algorithm::Dutch is the current one. Algorithm::DutchPorter is the earlier algorithm, kept for indexes built with it, since the two produce different stems for about half of the Dutch test vocabulary.

License

The runtime and API code is MIT licensed, see LICENSE. The per-language algorithm modules are produced by the Snowball compiler from the Snowball project's algorithm sources, which are distributed under the BSD-3-Clause license. Its copyright and notice ship in LICENSE-BSD, and the package license is MIT AND BSD-3-Clause. The shipped test data for thirteen languages is sampled from Snowball test vocabularies covered by that same BSD-3-Clause notice. Some Snowball vocabularies carry other licenses, so for Arabic, Czech, Finnish, French, Greek, Italian, Polish, and Tamil the shipped test data instead pairs original word lists written for this crate with stems computed by the Snowball project's stemwords tool, and contains no text from those vocabularies.