Struct StreamingColumnCollection 

pub struct StreamingColumnCollection { /* private fields */ }

Implementations

impl StreamingColumnCollection

pub fn new() -> Self

pub fn memory_limit(limit_mb: usize) -> Self

pub fn with_semantic_hints(self, hints: &SemanticHints) -> Self

Configure value-driven semantic hints before records are processed.

pub fn init_columns(&mut self, headers: &[String])

pub fn init_column_with_missing(&mut self, header: &str, prior_rows: usize)

Add a column discovered after prior_rows records have already passed.

JSON objects may introduce keys at any point in a stream. Those earlier objects are missing the new key, so the column must start with matching total/null counters rather than looking shorter and more complete than the dataset.

pub fn process_record<I>(&mut self, headers: &[String], values: I)

where I: IntoIterator<Item = String>,

pub fn process_record_with_containers<I>( &mut self, headers: &[String], values: I, containers: &[usize], )

where I: IntoIterator<Item = String>,

process_record for a record whose fields at containers (positions in headers) were JSON objects or arrays, given as their text. Their statistics read the text like any other value; only the row signature tells them apart from a string that spells the same text (#945).

pub fn row_duplicate_summary(&self) -> Option<RowDuplicateSummary>

Full-stream row-duplicate counts, or None when no rows were seen.

pub fn row_completeness_summary(&self) -> Option<RowCompletenessSummary>

Full-stream complete-record counts, or None when no rows were seen.

pub fn semantic_hint_bindings(&self) -> Vec<SemanticHintBinding>

Exact value-driven semantic-hint evidence over every processed record.

pub fn get_column_stats( &self, column_name: &str, ) -> Option<&StreamingStatistics>

pub fn column_names(&self) -> Vec<String>

pub fn retain_columns(&mut self, names: &[String])

Retain only the selected columns, preserving their source order.

Parsers call this after validating the selection against the complete source schema. Row-level trackers intentionally remain unchanged; a projected report withholds the row-level quality dimensions because their meaning changes when columns are removed.

pub fn memory_usage_bytes(&self) -> usize

pub fn is_memory_pressure(&self) -> bool

pub fn reduce_memory_usage(&mut self)

pub fn column_type_fingerprint(&self) -> u64

Fingerprint of each column’s currently inferred data type.

Returns a u64 hash suitable for cheap comparison in a schema stability tracker.

pub fn merge(&mut self, other: StreamingColumnCollection)

Fold another collection into this one, as if a single pass had read both sides.

The merged profile describes the union of the two schemas, and three consequences of that are each a silently wrong number when skipped:

• A column only one side saw has to join ordered_names, or Self::column_names omits it and the column vanishes from the report while its statistics sit unreachable in the map.
• That column is absent from every record the other side read, so it is padded with those rows as missing values, the way Self::init_column_with_missing pads a key that appears mid-stream. Skipping the padding leaves the column looking shorter and more complete than the dataset.
• Rows counted complete against the narrower schema are not complete against the union, so that side’s complete-row count is invalidated for the same reason.

Row signatures are built per record from that record’s own headers, and encode values only. Two sides that read different columns, or the same columns in a different order, therefore sign different rows identically. Those signatures cannot be rebuilt against the union schema once folded into the estimator, so the merged duplicate count is withheld rather than reported wrong.

Trait Implementations

impl Default for StreamingColumnCollection

fn default() -> Self

Returns the “default value” for a type.