Struct hadean_std::dataframe::DataFrame

pub struct DataFrame<T> where T: ProcessSendable {
    // some fields omitted
}

A scalable distributed dataframe datastructure. DataFrame<T> is a dataframe datastructure that holds a collection of elements of type T.

It enables computation to be performed in parallel across many elements at the same time.

Here's an example showing addition of data to a DataFrame, and map, filter, and reduce being called upon it:

use hadean_std::dataframe::DataFrame;

let mut data_frame: DataFrame<String> = DataFrame::new();
data_frame.push(String::from("http://google.com"));
data_frame.push(String::from("http://bbc.co.uk"));
data_frame.push(String::from("https://hadean.com"));
let mut data_frame: DataFrame<String> = data_frame.map(|mut val|{val.push_str("/xyz");val});
data_frame.filter(|val|val.len() > 20);
data_frame.reduce(0u64, |acc, val| acc + val.len() as u64);

Methods

impl<T> DataFrame<T> where T: ProcessSendable

fn new() -> DataFrame<T>

Constructs a new, empty DataFrame<T>.

fn push(&mut self, val: T)

Append val to the dataframe.

fn map<F, T1>(self, f: F) -> DataFrame<T1> where F: Fn(T) -> T1 + ProcessSendable, T1: ProcessSendable

Map each element of the dataframe using f.

fn filter<F>(&mut self, f: F) where F: Fn(&T) -> bool + ProcessSendable

Filter elements of the dataframe, keeping only those that satisfy f.

fn reduce<F, T1>(&self, initial: T1, f: F) -> T1 where F: Fn(T1, &T) -> T1 + ProcessSendable, T1: ProcessSendable

Reduce the dataframe into one value.

Examples

let mut data_frame: DataFrame<String> = DataFrame::new();
data_frame.push(String::from("http://google.com"));
data_frame.push(String::from("http://bbc.co.uk"));
data_frame.push(String::from("https://hadean.com"));
let total_length = data_frame.reduce(0, |acc, val| -> acc + val.len());