snowstem 0.1.0

Snowball stemming for 21 languages, generated from the current Snowball release, with no required dependencies and no_std support
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

# snowstem

Snowball stemming for 21 languages.

Each algorithm module is produced by the official Snowball compiler,
version 3.1.1, and checked word for word against the Snowball project's
test vocabularies. The crate has no required dependencies, builds
without `std`, and stems a word with at most one allocation. The
minimum supported Rust version is 1.63.

## Usage

```rust
use snowstem::{Algorithm, Stemmer};

let stemmer = Stemmer::create(Algorithm::English);
assert_eq!(stemmer.stem("international"), "internat");
assert_eq!(stemmer.stem("internal"), "internal");
```

Input is expected in lower case. Snowball defines stemming over
lowercase words, so casefold and tokenize before calling.
`Stemmer::stem_into` writes the stem into a `String` you own, so a loop
over many words stops allocating once the buffer fits the longest word.

The opt-in `serde` feature adds `Serialize` and `Deserialize` for
`Algorithm` as its lowercase name. The default build depends on no
other crate.

## Languages

Arabic, Czech, Danish, Dutch, English, Finnish, French, German, Greek,
Hungarian, Italian, Norwegian, Polish, Portuguese, Romanian, Russian,
Spanish, Swedish, Tamil, and Turkish.

Snowball 3.0 replaced its Dutch algorithm. `Algorithm::Dutch` is the
current one. `Algorithm::DutchPorter` is the earlier algorithm, kept for
indexes built with it, since the two produce different stems for about
half of the Dutch test vocabulary.

## License

The runtime and API code is MIT licensed, see [LICENSE](LICENSE). The
per-language algorithm modules are produced by the Snowball compiler
from the Snowball project's algorithm sources, which are distributed
under the BSD-3-Clause license. Its copyright and notice ship in
[LICENSE-BSD](LICENSE-BSD), and the package license is
`MIT AND BSD-3-Clause`. The shipped test data for thirteen languages
is sampled from Snowball test vocabularies covered by that same
BSD-3-Clause notice. Some Snowball vocabularies carry other licenses,
so for Arabic, Czech, Finnish, French, Greek, Italian, Polish, and
Tamil the shipped test data instead pairs original word lists written
for this crate with stems computed by the Snowball project's
`stemwords` tool, and contains no text from those vocabularies.