Struct Meter 

pub struct Meter { /* private fields */ }

What the open on screen cost, as it is measured.

Shared with the threads that do the listing and the footer reads, which is why it is held behind an Arc and written through &self.

There is no way to clear one. Opening a dataset builds a new meter instead, for the reason the footer counter does: abandoning a load cancels nothing, so the reads of the directory that was walked away from are still running, and a meter they still held would go on adding their figures to the next dataset’s.

Implementations

impl Meter

pub fn listed(&self, took: Duration, files: Option<usize>, over_the_wire: bool)

Finding the dataset’s files took took and returned files of them.

over_the_wire is always false here today, and the parameter is kept so the two stretches record the same way. No listing route can count its requests: a local walk makes none, and every remote one — the flat list and the level-by-level walk a glob uses alike — hands the paging to the object store, which does not say how many round trips it took.

pub fn read_footers( &self, took: Duration, files: Option<usize>, over_the_wire: bool, )

A pass over files footers took took.

over_the_wire publishes the requests counted by Self::footer_request since the meter was made. A pass that read from a disk passes false: there were no requests, and a zero would read as none having been needed.

pub fn counted_rows( &self, took: Duration, files: Option<usize>, wire: Option<OverTheWire>, ) -> bool

A pass over files footers, run to settle the dataset’s row count, took took.

Recorded once and then never again, and the return says which happened. Counting runs whenever the row count is invalidated — clearing a filter does it, so a few minutes of exploring runs it several times — and those later passes are re-work on a dataset that is already open. Adding them would make a section headed by what opening the dataset cost climb for as long as the session lasted.

pub fn read_page(&self, took: Duration, files: Option<usize>)

Reading the page now on screen took took and read files of the dataset’s files.

Replaces rather than adds: this is the cost of the page a user is looking at, and adding every page they have scrolled through would answer a question nobody asked. files is None where Polars was handed the whole scan and decided for itself what to read.

No byte figure. Polars does this read and does not report what it fetched, and the row-group sizes the footers hold would give the size of whole row groups for every column — not the columns on screen, and not what crossed the wire. A number that wrong is worse than none.

pub fn last_page(&self) -> Option<Cost>

What the page now on screen cost, or None before one has been read.

pub fn footer_request(&self, bytes: u64)

One footer request, and the bytes it returned. Counted as the reads happen; what they come to is published when the pass ends.

pub fn listing(&self) -> Option<Cost>

What finding the files cost, or None if no listing has been measured.

pub fn footers(&self) -> Option<Cost>

What reading the footers cost, or None if no footer pass has been measured.

pub fn total(&self) -> Option<Total>

Everything measured so far, added up. None until something has been measured.

No file count. The stretches count different things — the listing counts the dataset’s files, the footer pass counts footers read, which on a staged open is more than there are files — so adding them gives a number that is not the size of anything, sitting under the same word the listing row uses for the size of the dataset.

The wire figures are the sum of the stretches that counted them, and are None when no stretch did: a local dataset’s total is a time, not a time and a pretence of nothing having been transferred.

Trait Implementations

impl Debug for Meter

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Meter

fn default() -> Self

Returns the “default value” for a type.