# vader-parity
VADER sentiment scoring for Rust, with exact output parity against Python
[vaderSentiment] 3.3.2.
`polarity_scores` returns four numbers per text: `neg`, `neu`, `pos`, and
`compound`. Each equals the value the Python package returns for the same
text, at that package's published precision: three decimals for the
proportions, four for the compound. The whole rule set ships. That covers
lexicon scoring, booster and dampener modifiers with the casing increment,
negation with the "least" special case, contrastive "but" reweighting,
exclamation and question mark emphasis, the special-case idioms, emoji
translation, and the alpha-15 normalization to compound, all over the
reference's own data files, vendored byte for byte from the 3.3.2 sdist.
Parity is the tested contract. The 177 frozen oracle vectors in
`tests/vectors/frozen.tsv` were captured from the pinned Python package
before implementation, and `tests/vectors.rs` replays them on every
`cargo test` with zero tolerated divergence. Before a release, a
differential harness scores a 63,471-sentence corpus against the live
pinned package under the same zero-divergence bar, and on that corpus the
pre-rounding compound is bit-identical to the oracle's. Casing is part of
the pin: `str.isupper()` and `str.lower()` are answered from
`data/casing_tables.tsv`, recorded from CPython 3.14.6 with Unicode data
16.0.0, and a sweep test checks every codepoint against the recorded
behavior, so the scores cannot drift when the Rust standard library moves
to a newer Unicode version than the reference interpreter's.
```toml
[dependencies]
vader-parity = "0.1.0"
```
## Use
```rust
use vader_parity::SentimentIntensityAnalyzer;
let analyzer = SentimentIntensityAnalyzer::new();
let scores = analyzer.polarity_scores("VADER is smart, handsome, and funny.");
assert_eq!(scores.neg, 0.0);
assert_eq!(scores.neu, 0.254);
assert_eq!(scores.pos, 0.746);
assert_eq!(scores.compound, 0.8316);
```
`polarity_scores` is total: every `&str` returns scores, and text with
nothing scoreable is neutral, exactly as the reference scores it (`is?`
returns a compound of 0.0 and a `neu` of 1.0, and the doctest on
`polarity_scores` asserts both). Construction parses the vendored lexicons
once, about 11,000 data lines, so build one analyzer and score many texts.
Each call is one bounded linear pass, and a doctest in the crate docs pins
that by scoring 10,000 sentences on one analyzer inside a fixed budget.
## Replacing `qsv_vader_sentiment_analysis`
This crate replaces the `qsv_vader_sentiment_analysis` /
`vader-sentimental` / `vader_sentiment` family, whose scoring defects
reverse the sign of `compound` on ordinary sentences. Migration is the
dependency line plus the return shape: where those crates hand back a map or
struct, read `scores.compound`, `scores.neg`, `scores.neu`, `scores.pos`
here. Scores change on migration wherever the incumbent was wrong, which is
the point. Two before/after examples, both pinned as regression tests named
`defect_m1_*` and `defect_m2_*` in `tests/vectors.rs`:
| `he is at least smart` | +0.4019 | -0.3089 |
| `these are the shittiest of times` | -0.6597 | positive: "shittiest" matched "the shit" (+3.0) by substring |
## Scope
- English only: the reference is English-only and so is its lexicon.
- Scoring engine only: sentence splitting, CSV integration, and custom
lexicons are out of scope (custom lexicons are the first candidate for a
minor release).
- Reference version 3.3.2 exactly: a reference upgrade would be a minor
release with a re-frozen corpus.
- `std` only, with zero runtime dependencies.
## Requirements
Rust 1.81.0 or later. The manifest declares `rust-version = "1.81.0"`, and a
CI job builds the library and runs the doctests on that exact toolchain.
## Attribution
VADER is the work of C.J. Hutto and E.E. Gilbert: the algorithm design, the
sentiment lexicon `vader_lexicon.txt`, and the emoji lexicon
`emoji_utf8_lexicon.txt`, published under the MIT license, copyright C.J.
Hutto. The idiom and booster tables in `data/rules_tables.tsv` are
transcribed from the reference source, and the two lexicon files are
byte-identical copies from the vaderSentiment 3.3.2 sdist. `data/PROVENANCE`
records the sha256 digest of the sdist and of every vendored file, and
`tests/provenance.rs` enforces them. The Rust implementation and the test
suite are this crate's own work.
If you use VADER, cite:
> Hutto, C.J. & Gilbert, E.E. (2014). VADER: A Parsimonious Rule-based Model
> for Sentiment Analysis of Social Media Text. Eighth International
> Conference on Weblogs and Social Media (ICWSM-14). Ann Arbor, MI, June
> 2014.
## License
The crate's own code is licensed under either of
[Apache License, Version 2.0](LICENSE-APACHE) or the
[MIT license](LICENSE-MIT) at your option.
The vendored data files in `data/` remain under the VADER authors' MIT
license regardless of which arm you pick for the code. That composition
works because MIT's obligation is notice preservation, and the crate ships
[`data/LICENSE-VADER`](data/LICENSE-VADER) in every archive. The manifest's
SPDX expression states exactly that: `(MIT OR Apache-2.0) AND MIT`.
Unless you state otherwise, any contribution you intentionally submit for
inclusion in this crate, as defined in the Apache-2.0 license, is dual
licensed as above, with no additional terms or conditions.
[vaderSentiment]: https://pypi.org/project/vaderSentiment/3.3.2/