Skip to main content

TableBuilder

Struct TableBuilder 

Source
pub struct TableBuilder { /* private fields */ }
Expand description

Generic columnar table builder. Implements ColumnarSink so any decoder can feed it field/value events; at the end it produces an Arrow RecordBatch.

Implementations§

Source§

impl TableBuilder

Source

pub fn new() -> Self

Source

pub fn with_capacity(cap: usize) -> Self

Source

pub fn with_plan(cap: usize, plan: Arc<ExecutionPlan>) -> Self

Source

pub fn ensure_schema(&mut self, schema: &FrozenSchema) -> Result<()>

Pre-size all columns from a FrozenSchema.

Called by parallel streaming workers so that every chunk’s TableBuilder has the full column set from construction, regardless of which fields appear in that chunk.

Source

pub fn num_rows(&self) -> usize

Source

pub fn num_columns(&self) -> usize

Source

pub fn column_names(&self) -> &[String]

Source

pub fn column_diagnostics(&self) -> Vec<(String, usize, usize)>

Diagnostic: (name, bytes_used, bytes_capacity) for each column.

Source

pub fn estimated_rows(&self) -> usize

The estimated_rows capacity hint passed to with_plan.

Source

pub fn finish(&mut self) -> Result<RecordBatch>

Finalize the builder into an Arrow RecordBatch.

This is also available as the ColumnarSink::finish trait method.

Source

pub fn reset(&mut self)

Reset all data while preserving the plan and estimated rows.

Source

pub fn normalize(&mut self)

Truncate every column back to row_count, dropping any partial-row values from a mid-field EOF. Idempotent.

Source

pub fn auto_dict_upgrade(&mut self)

If auto_dict is set, upgrade low-cardinality string columns using the plan’s threshold/max-size tuning (defaults: 5% ratio, max size 256).

Source

pub fn sort_columns(&mut self)

Sort columns according to schema_order. Columns named in schema_order appear in that order; any other columns keep their relative first-appearance order after the ordered ones.

Source§

impl TableBuilder

Source

pub fn extend(&mut self, other: TableBuilder) -> Result<()>

Merge another builder’s data into this one, consuming other.

New columns discovered in other are null-padded for the existing rows in self. Columns present in self but absent from other are null-padded for other’s rows. Column order follows first-appearance order across both builders.

Columns that disagree on storage variant are reconciled with safe promotions (int64→float64, string→dictionary); irreconcilable conflicts return crate::Error::Merge.

Trait Implementations§

Source§

impl ColumnarSink for TableBuilder

Source§

fn put_field_at(&mut self, slot: u32, value: Value<'_>)

Push a value directly to a known slot index, bypassing all name resolution and hash lookups. Used by adapters that verified the field identity via expect_slot + memcmp on raw bytes.

Source§

fn begin_row(&mut self)

Source§

fn put_field(&mut self, name: &str, value: Value<'_>)

Source§

fn end_row(&mut self)

Source§

fn wants(&self, name: &str) -> bool

Return false to signal that the engine will drop this field.
Source§

fn resolve<'a>(&'a self, name: &'a str) -> Option<&'a str>

Resolve a raw field name to its output column name, or None if dropped. Default keeps the name as-is. Adapters that do expensive extraction can call this once and then put_field_resolved to avoid a second resolve_field inside put_field.
Source§

fn put_field_resolved(&mut self, resolved_name: &str, value: Value<'_>)

Push a field that has already been resolved via resolve. Default delegates to put_field (which will resolve again). Overrides should bypass the rename/drop lookup.
Source§

fn resolve_and_put(&mut self, name: &str, value: Value<'_>)

Resolve a field name and push it in one call, avoiding the double resolve_field that wants + put_field would perform. Default: resolve then put_field_resolved (with owned clone to satisfy the borrow checker). Overrides should avoid the allocation.
Source§

fn row_rejected(&self) -> bool

Whether the current row has been rejected by the filter and the scanner should skip the remainder of the row. Used for predicate-first deferred materialization: once Fail, the scanner can byte-jump to </Details> without further field scanning. Default false.
Source§

fn row_satisfied(&self) -> bool

Whether every wanted column for the current row already has a value. The adapter can byte-jump to the row close tag when this returns true, skipping remaining fields. Composes with row_rejected (a row can be satisfied OR rejected, both short-circuit). Read more
Source§

fn wanted_mask(&self) -> u64

Bitmask of columns that the projection wants. Bit i is set iff column i is in the output schema. The adapter can test membership with (wanted_mask >> slot) & 1 == 1 instead of a virtual call per field. Returns empty mask (all zeros) when no projection is active.
Source§

fn finish(&mut self) -> Result<RecordBatch>

Finalize the sink into an Arrow RecordBatch.
Source§

fn expect_slot(&self, ordinal: u32) -> Option<(u32, &[u8])>

After the layout is learned, the expected (slot, raw name bytes) at this ordinal position in the record. The adapter can compare the raw bytes in-place (one memcmp) instead of running the full attribute scan → UTF-8 decode → hash → lookup path. Read more
Source§

fn record_slot(&mut self, ordinal: u32, slot: u32, raw_name: &[u8])

Adapter reports the slot it resolved for ordinal so the engine can cache it for subsequent rows. Called after the adapter resolves a field via the generic path (wants + resolve + put_field_resolved).
Source§

fn layout_broken(&mut self, ordinal: u32)

Adapter reports that the layout expected at ordinal did not match. The engine should invalidate its cached expectation and fall back to generic resolution for subsequent rows.
Source§

fn reset_child_ordinal(&mut self)

Reset the ordinal counter for child-element tracking. Called after row-tag attributes are emitted so child ordinals start at 0.
Source§

fn put_row(&mut self, fields: &[(&str, Value<'_>)])

Push a complete row. Read more
Source§

fn resolve_raw<'a>(&'a self, raw_name: &'a [u8]) -> Option<&'a str>

Resolve a valid raw name without allocating a UTF-8 String.
Source§

fn resolve_and_put_raw(&mut self, raw_name: &[u8], value: Value<'_>)

Push a value for a field whose name is still in its raw byte form.
Source§

fn needs_value(&self) -> bool

Whether this sink needs decoded values from the parser. Read more
Source§

impl Default for TableBuilder

Source§

fn default() -> Self

Returns the “default value” for a type. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Allocation for T
where T: RefUnwindSafe + Send + Sync,

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.