Struct WordFreq

pub struct WordFreq { /* private fields */ }

Implementation of wordfreq.

Implementations

impl WordFreq

pub fn new<I, W>(word_weights: I) -> Self

where I: IntoIterator<Item = (W, Float)>, W: AsRef<str>,

Creates an instance from frequencies.

Arguments

• word_weights: Pairs of words and their frequencies (or probabilities) from a corpus.

Notes

If the input contains duplicate words, the last occurrence is used.

pub fn minimum(self, minimum: Float) -> Result<Self>

Sets the lower bound of returned frequencies (default is 0.0).

An error is returned if the input is negative.

pub fn standardizer(self, standardizer: Standardizer) -> Self

Sets the standardizer for preprocessing words.

If set, the standardizer is always applied to words before looking up their frequencies.

pub fn word_frequency<W>(&self, word: W) -> Float

where W: AsRef<str>,

Returns the word’s frequency, normalized between 0.0 and 1.0.

Examples

use approx::assert_relative_eq;
use wordfreq::WordFreq;

let word_weights = [("las", 10.), ("vegas", 30.)];
let wf = WordFreq::new(word_weights);

assert_relative_eq!(wf.word_frequency("las"), 0.25);
assert_relative_eq!(wf.word_frequency("vegas"), 0.75);
assert_relative_eq!(wf.word_frequency("Las"), 0.00);

pub fn zipf_frequency<W>(&self, word: W) -> Float

where W: AsRef<str>,

Returns the Zipf frequency of a word as a human-friendly logarithmic scale.

Examples

use approx::assert_relative_eq;
use wordfreq::WordFreq;

let word_weights = [("las", 10.), ("vegas", 30.)];
let wf = WordFreq::new(word_weights);

assert_relative_eq!(wf.zipf_frequency("las"), 8.4);
assert_relative_eq!(wf.zipf_frequency("vegas"), 8.88);
assert_relative_eq!(wf.zipf_frequency("Las"), 0.00);

pub fn serialize(&self) -> Result<Vec<u8>>

Exports the model data.

Note that the format is distinct from the one used in the oritinal Python package.

pub fn deserialize(bytes: &[u8]) -> Result<Self>

Deserializes the model, which is exported by WordFreq::serialize().

pub const fn word_frequency_map(&self) -> &HashMap<String, Float>

Returns the reference to the internal word-frequency map.

Trait Implementations

impl Clone for WordFreq

fn clone(&self) -> WordFreq

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.