Struct DatasetShape 

pub struct DatasetShape {
    pub fingerprint: String,
    pub files: Vec<CachedFooter>,
    pub schemas: Vec<Vec<(String, DataType)>>,
    pub taken_at: u64,
}

What an open learned about a dataset’s files, so the next open shows columns and row count without reading footers (seconds for thousands of objects). The listing, which happens anyway, decides via the fingerprint whether this still holds. A cache, not a catalog, like DatasetFacts.

Fields

fingerprint: String

What the files looked like when taken; a listing with a different one describes a changed dataset, and the rest is ignored.

files: Vec<CachedFooter>

One entry per file, in listing order, which is scan order.

schemas: Vec<Vec<(String, DataType)>>

The distinct schemas, by index, since thousands of files usually share one. Types are Polars’ own serialization (names cannot rebuild nested types); if a Polars upgrade changes it, entries stop parsing and the cache is just empty.

taken_at: u64

Seconds since the Unix epoch, for a human reading the file.

Implementations

impl DatasetShape

pub fn fingerprint_of<'a>( files: impl IntoIterator<Item = (&'a str, u64, u64, Option<&'a str>)>, ) -> String

A listing’s fingerprint: file names, count, sizes, mtimes and store tags, all seen without opening anything. Names are included since remembered footers join to a fresh listing by position; ETags since size and whole-second mtime miss same-length rewrites within a second.

pub fn intern_schema( schemas: &mut Vec<Vec<(String, DataType)>>, schema: &Schema, ) -> usize

Where schema sits in schemas, added on the end if it is not there yet.

pub fn schema_at( schemas: &[Vec<(String, DataType)>], at: usize, ) -> Option<Schema>

The schema at at in schemas, or None when the table has no such entry.

Trait Implementations

impl Clone for DatasetShape

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DatasetShape

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for DatasetShape

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for DatasetShape

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Eq for DatasetShape

impl PartialEq for DatasetShape

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for DatasetShape

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for DatasetShape