nltk-porter 0.1.0

NLTK's Porter stemmer ported to Rust, with the ORIGINAL / MARTIN / NLTK extension modes
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

# nltk-porter

[![crates.io](https://img.shields.io/crates/v/nltk-porter.svg)](https://crates.io/crates/nltk-porter)

A faithful Rust port of NLTK's Porter stemmer (`nltk.stem.porter.PorterStemmer`),
dependency-free and verified against NLTK.

NLTK ships its own Porter implementation with three selectable modes; this port
reproduces all three exactly (unlike `rust-stemmers`, which wraps Snowball's Porter).

```rust
use nltk_porter::{PorterStemmer, Mode};

let p = PorterStemmer::new(Mode::Nltk);
assert_eq!(p.stem("caresses"), "caress");
assert_eq!(p.stem("happy"), "happi");
assert_eq!(p.stem("skies"), "sky");
```

## Modes

- `Mode::Nltk` (default) — NLTK contributors' extensions.
- `Mode::Martin` — only the extensions on Martin Porter's website.
- `Mode::Original` — faithful to the 1980 paper (Porter deprecates this).

## Verification

Differential-tested against Python `nltk`'s `PorterStemmer`: 68,000+ words (the NLTK
`words` corpus plus tricky cases and random fuzz tokens) in each of the three modes,
zero mismatches. Words are processed by Unicode scalar value, matching Python's `str`.

Ported from NLTK (Apache-2.0).