vader-parity

VADER sentiment scoring for Rust, with exact output parity against Python vaderSentiment 3.3.2.

polarity_scores returns four numbers per text: neg, neu, pos, and compound. Each equals the value the Python package returns for the same text, at that package's published precision: three decimals for the proportions, four for the compound. The whole rule set ships. That covers lexicon scoring, booster and dampener modifiers with the casing increment, negation with the "least" special case, contrastive "but" reweighting, exclamation and question mark emphasis, the special-case idioms, emoji translation, and the alpha-15 normalization to compound, all over the reference's own data files, vendored byte for byte from the 3.3.2 sdist.

Parity is the tested contract. The 177 frozen oracle vectors in tests/vectors/frozen.tsv were captured from the pinned Python package before implementation, and tests/vectors.rs replays them on every cargo test with zero tolerated divergence. Before a release, a differential harness scores a 63,471-sentence corpus against the live pinned package under the same zero-divergence bar, and on that corpus the pre-rounding compound is bit-identical to the oracle's. Casing is part of the pin: str.isupper() and str.lower() are answered from data/casing_tables.tsv, recorded from CPython 3.14.6 with Unicode data 16.0.0, and a sweep test checks every codepoint against the recorded behavior, so the scores cannot drift when the Rust standard library moves to a newer Unicode version than the reference interpreter's.

[dependencies]
vader-parity = "0.1.0"

Use

use vader_parity::SentimentIntensityAnalyzer;

let analyzer = SentimentIntensityAnalyzer::new();
let scores = analyzer.polarity_scores("VADER is smart, handsome, and funny.");
assert_eq!(scores.neg, 0.0);
assert_eq!(scores.neu, 0.254);
assert_eq!(scores.pos, 0.746);
assert_eq!(scores.compound, 0.8316);

polarity_scores is total: every &str returns scores, and text with nothing scoreable is neutral, exactly as the reference scores it (is? returns a compound of 0.0 and a neu of 1.0, and the doctest on polarity_scores asserts both). Construction parses the vendored lexicons once, about 11,000 data lines, so build one analyzer and score many texts. Each call is one bounded linear pass, and a doctest in the crate docs pins that by scoring 10,000 sentences on one analyzer inside a fixed budget.

Replacing qsv_vader_sentiment_analysis

This crate replaces the qsv_vader_sentiment_analysis / vader-sentimental / vader_sentiment family, whose scoring defects reverse the sign of compound on ordinary sentences. Migration is the dependency line plus the return shape: where those crates hand back a map or struct, read scores.compound, scores.neg, scores.neu, scores.pos here. Scores change on migration wherever the incumbent was wrong, which is the point. Two before/after examples, both pinned as regression tests named defect_m1_* and defect_m2_* in tests/vectors.rs:

Scope

• English only: the reference is English-only and so is its lexicon.
• Scoring engine only: sentence splitting, CSV integration, and custom lexicons are out of scope (custom lexicons are the first candidate for a minor release).
• Reference version 3.3.2 exactly: a reference upgrade would be a minor release with a re-frozen corpus.
• std only, with zero runtime dependencies.

Requirements

Rust 1.81.0 or later. The manifest declares rust-version = "1.81.0", and a CI job builds the library and runs the doctests on that exact toolchain.

Attribution

VADER is the work of C.J. Hutto and E.E. Gilbert: the algorithm design, the sentiment lexicon vader_lexicon.txt, and the emoji lexicon emoji_utf8_lexicon.txt, published under the MIT license, copyright C.J. Hutto. The idiom and booster tables in data/rules_tables.tsv are transcribed from the reference source, and the two lexicon files are byte-identical copies from the vaderSentiment 3.3.2 sdist. data/PROVENANCE records the sha256 digest of the sdist and of every vendored file, and tests/provenance.rs enforces them. The Rust implementation and the test suite are this crate's own work.

If you use VADER, cite:

Hutto, C.J. & Gilbert, E.E. (2014). VADER: A Parsimonious Rule-based Model for Sentiment Analysis of Social Media Text. Eighth International Conference on Weblogs and Social Media (ICWSM-14). Ann Arbor, MI, June 2014.

License

The crate's own code is licensed under either of Apache License, Version 2.0 or the MIT license at your option.

The vendored data files in data/ remain under the VADER authors' MIT license regardless of which arm you pick for the code. That composition works because MIT's obligation is notice preservation, and the crate ships data/LICENSE-VADER in every archive. The manifest's SPDX expression states exactly that: (MIT OR Apache-2.0) AND MIT.

Unless you state otherwise, any contribution you intentionally submit for inclusion in this crate, as defined in the Apache-2.0 license, is dual licensed as above, with no additional terms or conditions.