Struct RowSignature 

pub struct RowSignature { /* private fields */ }

Canonical encoding of one row, for duplicate detection.

Fields are length-prefixed so their boundaries are unambiguous: ["ab", "c"] and ["a", "bc"] must not sign alike. Trailing absent fields are then dropped, which makes the signature independent of how wide the schema had grown when the row was read. A JSON key discovered mid-stream appends a column, and without the trim every row that predates it signs one field shorter than an identical row read afterwards, so the two count as distinct and the duplicate total comes up silently short.

The trim is safe because an absent field and an explicitly empty one are the same null to this profiler: it collapses only rows that already hold equal values in every column they share. On a fixed schema it applies uniformly to every row, so it changes no count there.

Implementations

impl RowSignature

pub fn push_field(&mut self, value: &str)

Append one field. An absent field is the empty string, the same representation StreamingColumnCollection::process_record gives a missing trailing value and an Arrow null.

Only an exactly empty value counts as absent here, deliberately: the broader is_null_like_token rule that decides null_count also matches null, nan and whitespace, and trimming on it would collapse a row ending in one of those against a row that ends in no field at all, inventing a duplicate.

pub fn finish(self) -> String

The signature, with trailing absent fields dropped.

Trait Implementations

impl Debug for RowSignature

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for RowSignature

fn default() -> Self

Returns the “default value” for a type.