Module dataframe

DataFrame representation for Spark Connection

Re-exports

pub use crate::readwriter::DataFrameReader;

pub use crate::readwriter::DataFrameWriter;

pub use crate::readwriter::DataFrameWriterV2;

pub use crate::streaming::DataStreamReader;

pub use crate::streaming::DataStreamWriter;

pub use crate::streaming::OutputMode;

pub use crate::streaming::StreamingQuery;

pub use spark::aggregate::GroupType;

pub use spark::analyze_plan_request::explain::ExplainMode;

pub use spark::join::JoinType;

pub use spark::write_operation::SaveMode;

Structs

DataFrame

DataFrame is composed of a SparkSession referencing a Spark Connect enabled cluster, and a LogicalPlanBuilder which represents the unresolved spark::Plan to be submitted to the cluster when an action is called.

DataFrameNaFunctions

Functionality for statistic functions with DataFrame.

DataFrameStatFunctions

Functionality for working with missing data in DataFrame.