Struct LocalDataFrame 

pub struct LocalDataFrame {
    pub schema: Schema,
    pub data: Vec<Column>,
    pub n_threads: usize,
    /* private fields */
}

Represents a local data frame which contains data stored in a columnar format and a well-defined Schema. Is useful for data sets that fit into memory or for testing/debugging purposes.

Fields

schema: Schema

The Schema of this data frame

data: Vec<Column>

The data of this data frame, in columnar format

n_threads: usize

Number of threads for this computer

Implementations

impl LocalDataFrame

An implementation for a LocalDataFrame, inspired by the data frames used in pandas and R.

pub fn from_sor(file_name: &str, from: usize, len: usize) -> Self

Creates a new LocalDataFrame from the given file by reading it in parallel using the number of cores available on this machine. Only reads len bytes of the file, starting at the given byte offset from.

pub fn new(schema: &Schema) -> Self

Creates an empty LocalDataFrame from the given Schema. The LocalDataFrame is created with no rows, but the names of the columns in the given schema are cloned.

pub fn get_schema(&self) -> &Schema

Obtains a reference to the schema of this LocalDataFrame

pub fn add_column( &mut self, col: Column, name: Option<String>, ) -> Result<(), LiquidError>

Adds a Column to this LocalDataFrame with an optional name. Returns a LiquidError::NameAlreadyExists if the given name is not unique.

pub fn get(&self, col_idx: usize, row_idx: usize) -> Result<Data, LiquidError>

Get the Data at the given col_idx, row_idx offsets.

pub fn get_col_idx(&self, col_name: &str) -> Option<usize>

Get the index of the Column with the given col_name. Returns Some if a Column with the given name exists, or None otherwise.

pub fn col_name(&self, col_idx: usize) -> Result<Option<&str>, LiquidError>

Given a column index, returns its name

pub fn set_string( &mut self, col_idx: usize, row_idx: usize, data: String, ) -> Result<(), LiquidError>

Mutates the value in this DataFrame at the given col_idx, row_idx to be changed to the given data.

pub fn set_bool( &mut self, col_idx: usize, row_idx: usize, data: bool, ) -> Result<(), LiquidError>

Mutates the value in this DataFrame at the given col_idx, row_idx to be changed to the given data.

pub fn set_float( &mut self, col_idx: usize, row_idx: usize, data: f64, ) -> Result<(), LiquidError>

Mutates the value in this DataFrame at the given col_idx, row_idx to be changed to the given data.

pub fn set_int( &mut self, col_idx: usize, row_idx: usize, data: i64, ) -> Result<(), LiquidError>

Mutates the value in this DataFrame at the given col_idx, row_idx to be changed to the given data.

pub fn fill_row( &self, row_index: usize, row: &mut Row, ) -> Result<(), LiquidError>

Set the fields of the given Row struct with values from this DataFrame at the given row_index.

If the row does not have the same schema as this DataFrame, a LiquidError::TypeMismatch error will be returned.

pub fn add_row(&mut self, row: &Row) -> Result<(), LiquidError>

Add a Row at the end of this DataFrame.

If the row does not have the same schema as this DataFrame, a LiquidError::TypeMismatch error will be returned.

pub fn map<T: Rower>(&self, rower: T) -> T

Applies the given rower synchronously to every row in this LocalDataFrame

Since map takes an immutable reference to self, the rower can not mutate this DataFrame. If mutation is desired, the rower must create its own DataFrame internally, clone each Row from this DataFrame as it visits them, and mutate the cloned row during each visit.

pub fn pmap<T: Rower + Clone + Send>(&self, rower: T) -> T

Applies the given rower to every row sequentially in this DataFrame The rower is cloned n_threads times, according to the value of n_threads for this LocalDataFrame. Each rower gets operates on a chunk of this LocalDataFrame and are run in parallel.

Since pmap takes an immutable reference to self, the rower can not mutate this LocalDataFrame. If mutation is desired, the rower must create its own LocalDataFrame internally by building one up as it visit rows, and mutates that.

n_threads defaults to the number of cores available on this machine.

pub fn filter<T: Rower>(&self, rower: &mut T) -> Self

Creates a new LocalDataFrame by applying the given rower to every row sequentially in this LocalDataFrame and cloning rows for which the given rower returns true from its accept method. Is run synchronously.

pub fn pfilter<T: Rower + Clone + Send>(&self, rower: &mut T) -> Self

Creates a new LocalDataFrame by applying the given rower to every row in this data frame sequentially, and cloning rows for which the given rower returns true from its accept method. The rower is cloned n_threads times, according to the value of n_threads for this LocalDataFrame. Each rower gets operates on a chunk of this LocalDataFrame and are run in parallel.

n_threads defaults to the number of cores available on this machine.

pub fn combine(self, other: Self) -> Result<Self, LiquidError>

Consumes this LocalDataFrame and the other given LocalDataFrame, returning a combined LocalDataFrame if successful.

• The columns names and the number of threads for the resulting LocalDataFrame are from this LocalDataFrame and the column names and n_threads in other are ignored
• The data of other is appended to the data of this LocalDataFrame

Errors

If the schema of this LocalDataFrame and other have different DataTypes

pub fn n_rows(&self) -> usize

Return the number of rows in this DataFrame.

pub fn n_cols(&self) -> usize

Return the number of columns in this DataFrame.

Trait Implementations

impl Clone for LocalDataFrame

fn clone(&self) -> LocalDataFrame

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LocalDataFrame

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl DeepSizeOf for LocalDataFrame

fn deep_size_of_children(&self, context: &mut Context) -> usize

Returns an estimation of the heap-managed storage of this object. This does not include the size of the object itself.

fn deep_size_of(&self) -> usize

Returns an estimation of a total size of memory owned by the object, including heap-managed storage.

fn recurse_deep_size_of(&self, context: &mut Context) -> usize

👎Deprecated since 0.1.1: use std::mem::size_of_val(val) + val.deep_size_of_children() instead

Deprecated: equivalent to std::mem::size_of_val(val) + val.deep_size_of_children()

impl<'de> Deserialize<'de> for LocalDataFrame

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Display for LocalDataFrame

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<Column> for LocalDataFrame

fn from(column: Column) -> Self

Construct a new DataFrame with the given column.

impl From<Data> for LocalDataFrame

fn from(scalar: Data) -> Self

Construct a new DataFrame with the given scalar value.

impl From<Vec<Column>> for LocalDataFrame

fn from(data: Vec<Column>) -> Self

Construct a new DataFrame with the given columns.

impl PartialEq for LocalDataFrame

fn eq(&self, other: &LocalDataFrame) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for LocalDataFrame

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for LocalDataFrame