Struct rammer::BagOfWords

pub struct BagOfWords { /* fields omitted */ }

A BagOfWords, also referred to as a bow, is a frequency map of words. Read more about the BagOfWords model here: BagOfWords Wikipedia. BagOfWords works with Unicode Words. Words are defined by as between UAX#29 word boundaries. BagOfWords is serializable using one of the serde serialization crates

use rammer::BagOfWords;
use serde_json;
let singly_trained_bow = BagOfWords::from_file("test_resources/test_data/unicode_and_ascii.txt").expect("File not found");
let big_bow = BagOfWords::from_folder("data/train/ham").expect("Folder not found");
let com_bow = singly_trained_bow.combine(big_bow);

Implementations

impl BagOfWords

pub fn new() -> Self

Return a new BagOfWords with an empty Frequency Map.

let empty_bow = BagOfWords::new();

pub fn from_file(file_path: &str) -> Option<Self>

Create a BagOfWords from a text file. This file should already be known to be ham or spam. The text file will be the basis of a new HSModel’s Ham/Spam BagOfWords

let spam_bow = BagOfWords::from_file("test_resources/test_data/unicode_and_ascii.txt").unwrap();

pub fn from_folder(dir_path: &str) -> Option<Self>

Create a BagOfWords from a folder containing either spam training text files, or ham training text files.

let spam_bow = BagOfWords::from_folder("data/train/spam");

pub fn combine(self, other: Self) -> Self

Combines two BagOfWords into a new BagOfWords. Freqencies of words found in both bags are additive. This operation is commutative and associative. These properties can be used to dynamically grow your training BagOfWords.

let ham_bow_1 = BagOfWords::from("Hello there world"); // Creates: {HELLO: 1, THERE: 1, WORLD: 1}
let ham_bow_2 = BagOfWords::from("howdy there guy"); // Creates: {HOWDY: 1, THERE: 1, GUY: 1}
let com_bow = ham_bow_1.combine(ham_bow_2); // Combines to: {HELLO: 1, THERE: 2, HOWDY: 1, ...}

pub fn total_word_count(&self) -> Count

Get the sum of all the Counts in a BagOfWords. Used internally for frequency calculations.

ham_bow.total_word_count(); // returns a sum of Counts.

pub fn word_frequency(&self, word: &str) -> Option<Frequency>

Calculates the Frequency of a word in the BagOfWords by taking count_of_a_word / total_word_count. This will return None, if the word slice passed contains multiple words.

let ham_bow = BagOfWords::from("hello there how are you");
ham_bow.word_frequency("hello"); //returns 0.2
ham_bow.word_frequency("hello there"); //returns None

Trait Implementations

impl Clone for BagOfWords

fn clone(&self) -> BagOfWords

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for BagOfWords

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for BagOfWords

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error> where
    __D: Deserializer<'de>, 

Deserialize this value from the given Serde deserializer.

impl From<&'_ str> for BagOfWords

Converts a &str to a bag of words. This to create BagOfWord models, consider using from_file or from_folder instead.

let bow = BagOfWords::from("hello world WOrLD"); // creates {HELLO: 1, WORLD: 2}

fn from(s: &str) -> BagOfWords

Performs the conversion.

impl FromIterator<BagOfWords> for BagOfWords

Use .collect() over an iterator of BagOfWords to additively combine them with combine

let bow: BagOfWords = vec![
    BagOfWords::from("hi"),
    BagOfWords::new(),
    BagOfWords::from("Big sale!")]
    .into_iter().collect();

fn from_iter<I: IntoIterator<Item = BagOfWords>>(iter: I) -> Self

Creates a value from an iterator.

impl FromParallelIterator<BagOfWords> for BagOfWords

Use .collect() over a parallel iterator of BagOfWords to additively combine them with combine use rayon crate to make .into_par_iter() available.

use rayon::prelude::*;
let bow: BagOfWords = vec![
    BagOfWords::from("hi"),
    BagOfWords::new(),
    BagOfWords::from("Big sale!")]
    .into_par_iter().collect();

fn from_par_iter<I>(par_iter: I) -> Self where
    I: IntoParallelIterator<Item = BagOfWords>, 

Creates an instance of the collection from the parallel iterator par_iter.

impl PartialEq<BagOfWords> for BagOfWords

fn eq(&self, other: &BagOfWords) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &BagOfWords) -> bool

This method tests for !=.

impl Serialize for BagOfWords

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error> where
    __S: Serializer, 

Serialize this value into the given Serde serializer.

impl Eq for BagOfWords

impl StructuralEq for BagOfWords

impl StructuralPartialEq for BagOfWords