Struct VarBinBuilder 

pub struct VarBinBuilder<O: OffsetBuilderPType> { /* private fields */ }

Builder for VarBinArray values with O-typed offsets.

This is the offset-based counterpart to VarBinViewBuilder: values are laid out contiguously in a single byte buffer described by a monotonically increasing offsets buffer. An i32 or i64 builder can therefore be handed straight to Arrow as a Utf8/LargeUtf8/Binary/ LargeBinary array without re-laying out the bytes.

Encodings that can decode into this layout should specialize append_to_builder with match_each_varbin_builder!, which recovers the concrete offset type from a &mut dyn ArrayBuilder so the decode loop is monomorphized over it.

Implementations

impl<O: OffsetBuilderPType> VarBinBuilder<O>

pub fn new(dtype: DType) -> Self

Creates an empty builder for dtype.

pub fn with_capacity(dtype: DType, capacity: usize) -> Self

Creates a builder for dtype with room for capacity values.

pub fn with_capacity_bytes(dtype: DType, capacity: usize, bytes: usize) -> Self

Creates a builder for dtype with room for capacity values totalling bytes bytes.

pub fn reserve_data(&mut self, additional: usize)

Reserves room for additional value bytes.

reserve_exact takes a row count and so cannot size the value bytes; callers that know the byte total should use this to size the buffer once instead of letting it grow.

pub fn append(&mut self, value: Option<&[u8]>)

Appends one value, or a null when it is None.

Panics

Panics if the resulting end offset does not fit in O. See append_value.

pub fn append_value(&mut self, value: impl AsRef<[u8]>)

Appends one non-null value.

Panics

Panics if the resulting end offset does not fit in O. The bulk appends report that as an error instead; use append_n_values for a fallible single append, or an i64 builder for byte totals past i32::MAX.

pub fn append_n_values( &mut self, value: impl AsRef<[u8]>, n: usize, ) -> VortexResult<()>

Appends the same non-null value n times.

Errors

Returns an error, leaving the builder unchanged, if the resulting end offsets do not fit in O.

pub fn push_null(&mut self)

Appends a null value.

Unlike append_null this does not check that the builder is nullable; the offsets and validity stay consistent either way, and a non-nullable dtype discards the validity bits on finish_into_varbin.

pub fn push_nulls(&mut self, n: usize)

Appends n null values. See push_null.

pub fn append_scalar_repeated( &mut self, scalar: &Scalar, n: usize, ) -> VortexResult<()>

Appends the same UTF-8 or binary scalar n times.

Errors

Returns an error if scalar has a different dtype than the builder, or if the resulting end offsets do not fit in O.

pub fn append_values<P>( &mut self, values: &[u8], end_offsets: impl Iterator<Item = P>, validity: &Mask, ) -> VortexResult<()>

where P: AsPrimitive<usize>, usize: AsPrimitive<O>,

Appends values from one contiguous byte buffer described by relative end offsets.

Each entry of end_offsets marks the end of one value relative to the start of values, and there must be exactly one per entry in validity.

pub unsafe fn append_decoded<P>( &mut self, num_bytes: usize, slack: usize, lengths: &[P], validity: &Mask, decode: &mut dyn FnMut(&mut [MaybeUninit<u8>]) -> VortexResult<usize>, ) -> VortexResult<()>

where P: AsPrimitive<usize>, usize: AsPrimitive<O>,

Appends values whose bytes decode writes straight into the builder’s byte storage.

decode is handed at least num_bytes + slack bytes of uninitialized storage and returns the number of bytes it initialized, which must be exactly num_bytes. Each entry of lengths is the byte length of one value — including nulls, which are zero-length — so there must be one per entry in validity.

slack is spare headroom past the values themselves. It exists so that a decoder which stores in wide fixed-size chunks can keep using them through the final value instead of dropping into a byte-at-a-time tail; a decoder must still never write past the slice it is handed, so 0 is correct for one that is already exact.

Decoding in place saves staging the decoded heap in a temporary buffer and copying it in.

decode is taken as a dyn reference so a caller that resolves the lengths type with match_each_integer_ptype! can build the closure once outside that match rather than inlining a whole decoder into each of its arms.

Errors

Returns an error, leaving the builder unchanged, if num_bytes would push an offset past what O can hold — checked before decode runs — or if decode reports a different byte count than num_bytes, or if lengths does not describe those bytes.

Safety

decode must initialize the first n bytes of the slice it is passed, where n is the value it returns. Those bytes are published as initialized without ever being read first, so a decode that over-reports leaves the builder holding uninitialized memory.

pub fn append_valid_slices<'a, F>( &mut self, num_bytes: usize, validity: &Mask, value: F, ) -> VortexResult<()>

where F: FnMut(usize) -> &'a [u8], usize: AsPrimitive<O>,

Appends validity.len() values, taking each valid row’s bytes from value.

value is called once per valid row in ascending row order and the slices it returns must total num_bytes bytes. Null rows consume no bytes and repeat the preceding offset.

The valid rows are walked a u64 word at a time rather than pulled one at a time through an iterator, and both buffers are sized once up front, so the copy loop costs one offset store plus one memcpy per value. A caller whose values only exist as a stream can still use this by pulling from the stream inside value: the rows are visited in ascending order.

pub fn append_varbin( &mut self, array: ArrayView<'_, VarBin>, ctx: &mut ExecutionCtx, ) -> VortexResult<()>

where usize: AsPrimitive<O>,

Appends a VarBinArray, reusing its offsets instead of walking its values.

pub fn append_varbinview( &mut self, array: ArrayView<'_, VarBinView>, ctx: &mut ExecutionCtx, ) -> VortexResult<()>

where usize: AsPrimitive<O>,

Appends a VarBinViewArray, compacting its values.

pub fn finish_into_varbin(&mut self) -> VarBinArray

Finishes the appended values into a VarBinArray and resets the builder.

Trait Implementations

impl<O: OffsetBuilderPType> ArrayBuilder for VarBinBuilder<O>

fn as_any(&self) -> &dyn Any

fn as_any_mut(&mut self) -> &mut dyn Any

fn dtype(&self) -> &DType

fn len(&self) -> usize

fn append_zeros(&mut self, n: usize)

Appends n “zero” values to the array.

unsafe fn append_nulls_unchecked(&mut self, n: usize)

The inner part of append_nulls.

fn append_scalar(&mut self, scalar: &Scalar) -> VortexResult<()>

A generic function to append a scalar to the builder.

fn reserve_exact(&mut self, additional: usize)

Allocate space for extra additional items

unsafe fn set_validity_unchecked(&mut self, validity: Mask)

override validity with the one provided, without checking lengths

fn finish(&mut self) -> ArrayRef

Constructs an Array from the builder components.

fn finish_into_canonical(&mut self, ctx: &mut ExecutionCtx) -> Canonical

Constructs a canonical array directly from the builder.

fn is_empty(&self) -> bool

fn append_zero(&mut self)

Append a “zero” value to the array.

fn append_null(&mut self)

Append a “null” value to the array.

fn append_nulls(&mut self, n: usize)

Appends n “null” values to the array.

fn append_default(&mut self)

Appends a default value to the array.

fn append_defaults(&mut self, n: usize)

Appends n default values to the array.

fn set_validity(&mut self, validity: Mask)

Override builders validity with the one provided.