Skip to main content

Crate vader_parity

Crate vader_parity 

Source
Expand description

VADER sentiment scoring with exact output parity against Python vaderSentiment 3.3.2.

VADER (Valence Aware Dictionary and sEntiment Reasoner) is the rule-based sentiment model of Hutto & Gilbert (ICWSM-14). This crate reimplements that package’s whole rule set over that package’s own data files, vendored byte-identical from the 3.3.2 sdist: lexicon scoring, booster and dampener modifiers with the casing increment, negation with the “least” special case, contrastive “but” reweighting, exclamation and question mark emphasis, the special-case idioms, emoji translation, and the alpha-15 normalization to compound. For every input, all four scores equal what the Python package returns at its published precision: three decimals for the proportions, four for the compound. The frozen oracle vectors in tests/vectors/frozen.tsv replay on every cargo test, and a release-time differential corpus against the pinned package holds that parity with zero tolerated divergence. Casing is pinned with the rest: str.isupper() and str.lower() answer from tables vendored out of CPython 3.14.6 with Unicode data 16.0.0, checked across every codepoint, so the scores hold still when the Rust standard library adopts a newer Unicode version than the reference interpreter’s.

use vader_parity::SentimentIntensityAnalyzer;

let analyzer = SentimentIntensityAnalyzer::new();
let scores = analyzer.polarity_scores("VADER is smart, handsome, and funny.");
assert_eq!(scores.neg, 0.0);
assert_eq!(scores.neu, 0.254);
assert_eq!(scores.pos, 0.746);
assert_eq!(scores.compound, 0.8316);

§Scoring many texts

SentimentIntensityAnalyzer::new parses the vendored lexicons, about 11,000 data lines, once. Build one analyzer and score many texts. Each call is a bounded linear pass, and the example below is the crate’s own regression tripwire for that claim.

use vader_parity::SentimentIntensityAnalyzer;

let analyzer = SentimentIntensityAnalyzer::new();
let started = std::time::Instant::now();
for _ in 0..10_000 {
    analyzer.polarity_scores("Today only kinda sux! But I'll get by, lol");
}
assert!(started.elapsed() < std::time::Duration::from_secs(30));

§Scope

The reference is English-only and so is its lexicon. This crate is the scoring engine only, pinned to reference version 3.3.2, std-only, with zero runtime dependencies. Sentence splitting, custom lexicons, and other reference versions are out of scope.

§Attribution

The VADER algorithm design, the sentiment lexicon, and the emoji lexicon are the MIT-licensed work of C.J. Hutto and E.E. Gilbert. The vendored files in data/ ship with the authors’ license text and their recorded sha256 digests, and the Rust implementation and its tests are this crate’s. If you use VADER, cite: Hutto, C.J. & Gilbert, E.E. (2014). VADER: A Parsimonious Rule-based Model for Sentiment Analysis of Social Media Text. Eighth International Conference on Weblogs and Social Media (ICWSM-14). Ann Arbor, MI.

Structs§

SentimentIntensityAnalyzer
The analyzer, holding the loaded lexicons.
SentimentScores
The four scores for one text. Field values match Python vaderSentiment 3.3.2, at that package’s published precision.