Skip to main content

analysis_rows

Function analysis_rows 

Source
pub fn analysis_rows(
    lf: &LazyFrame,
    sample_rows: Option<usize>,
    known_total: Option<usize>,
    seed: u64,
    polars_streaming: bool,
) -> Result<AnalysisRows>
Expand description

The rows an analysis works on: all when the table has at most sample_rows (or it is None), else a seeded sample spread across the table:

  • A plan whose slices reach into one Parquet or IPC scan reads SAMPLE_BLOCKS short runs at seeded places (a few dozen row groups of a huge table); known_total saves the count, else footers give it.
  • Anything else (filter, query, file union, CSV) streams once, keeping the lowest-ranked rows: uniform, bounded memory, counting rows in the same pass.