pub struct LazyFrame { /* private fields */ }Expand description
Lazy query API (LazyFrame, LazyGroupBy).
A lazily-evaluated query backed by a LogicalPlan.
Implementations§
Source§impl LazyFrame
impl LazyFrame
Sourcepub fn from_dataframe(df: DataFrame) -> Self
pub fn from_dataframe(df: DataFrame) -> Self
Create a LazyFrame that scans an in-memory DataFrame.
Sourcepub fn scan_csv(path: impl AsRef<Path>) -> Result<Self>
pub fn scan_csv(path: impl AsRef<Path>) -> Result<Self>
Build a CSV scan plan (no file I/O is performed until collect()).
Sourcepub fn scan_parquet(path: impl AsRef<Path>) -> Result<Self>
pub fn scan_parquet(path: impl AsRef<Path>) -> Result<Self>
Build a Parquet scan plan (no file I/O is performed until collect()).
Sourcepub fn concat(inputs: Vec<LazyFrame>) -> Result<Self>
pub fn concat(inputs: Vec<LazyFrame>) -> Result<Self>
Construct strict vertical concatenation from two or more lazy inputs.
When every schema is already known, mismatch is reported at plan construction. For lazy file scans whose schema is intentionally unavailable until bounded open, the streaming executor preflights every child before its first output and rejects any mismatch then.
Sourcepub fn select(self, exprs: Vec<Expr>) -> Self
pub fn select(self, exprs: Vec<Expr>) -> Self
Add a projection (select) node to the logical plan.
Sourcepub fn with_columns(self, exprs: Vec<Expr>) -> Self
pub fn with_columns(self, exprs: Vec<Expr>) -> Self
Add a projection (with_columns) node to the logical plan.
Sourcepub fn group_by(self, by: Vec<Expr>) -> LazyGroupBy
pub fn group_by(self, by: Vec<Expr>) -> LazyGroupBy
Start a group-by on this LazyFrame.
Sourcepub fn join<K: Into<JoinKeys>>(
self,
other: LazyFrame,
keys: K,
how: JoinType,
) -> Self
pub fn join<K: Into<JoinKeys>>( self, other: LazyFrame, keys: K, how: JoinType, ) -> Self
Join with another LazyFrame using provided join keys.
Sourcepub fn sort(self, options: SortOptions) -> Self
pub fn sort(self, options: SortOptions) -> Self
Sort by one or more columns.
Sourcepub fn fill_null<T: Into<FillNull>>(self, fill: T) -> Self
pub fn fill_null<T: Into<FillNull>>(self, fill: T) -> Self
Fill null values using a scalar or strategy.
Sourcepub fn drop_nulls(self, subset: Option<Vec<String>>) -> Self
pub fn drop_nulls(self, subset: Option<Vec<String>>) -> Self
Drop rows containing null values.
Sourcepub fn null_count(self) -> Self
pub fn null_count(self) -> Self
Count null values per column.
Sourcepub fn explode(self, column: impl Into<String>) -> Self
pub fn explode(self, column: impl Into<String>) -> Self
Explode one List<Utf8> column into multiple rows.
Sourcepub fn collect(self) -> Result<DataFrame>
pub fn collect(self) -> Result<DataFrame>
Optimize, compile, and execute this LazyFrame into an eager DataFrame.
Sourcepub fn collect_with_options(self, options: StreamOptions) -> Result<DataFrame>
pub fn collect_with_options(self, options: StreamOptions) -> Result<DataFrame>
Materialize a supported bounded stream while reserving every retained output batch.
This deliberately uses the streaming source/operator path and never wraps the eager executor with accounting after allocation.
Sourcepub fn collect_streaming(
self,
options: StreamOptions,
) -> Result<DataFrameStream>
pub fn collect_streaming( self, options: StreamOptions, ) -> Result<DataFrameStream>
Compile this plan for incremental bounded execution.
The compiler performs eligibility analysis before any source is opened. Until a source’s
v0.8 BatchSourceFactory is installed, it returns StreamingUnsupported rather than
calling the legacy eager executor.