Struct FixedSizeBinaryDictionaryBuilder 

pub struct FixedSizeBinaryDictionaryBuilder<K>
where
    K: ArrowDictionaryKeyType,
{ /* private fields */ }

Builder for DictionaryArray of FixedSizeBinaryArray

The output array has a dictionary of unique, fixed-size binary values. The builder handles deduplication.

Example

// Build 3 byte FixedBinaryArrays
let byte_width = 3;
let mut builder = FixedSizeBinaryDictionaryBuilder::<Int8Type>::new(3);
builder.append("abc").unwrap();
builder.append_null();
builder.append(b"def").unwrap();
builder.append(b"def").unwrap(); // duplicate value
// Result is a Dictionary Array
let array = builder.finish();
let dict_array = array.as_any().downcast_ref::<DictionaryArray<Int8Type>>().unwrap();
// The array represents "abc", null, "def", "def"
assert_eq!(array.keys().len(), 4);
// but there are only 2 unique values
assert_eq!(array.values().len(), 2);
let values = dict_array.values().as_any().downcast_ref::<FixedSizeBinaryArray>().unwrap();
assert_eq!(values.value(0), "abc".as_bytes());
assert_eq!(values.value(1), "def".as_bytes());

Implementations

impl<K> FixedSizeBinaryDictionaryBuilder<K>

where K: ArrowDictionaryKeyType,

pub fn new(byte_width: i32) -> FixedSizeBinaryDictionaryBuilder<K>

Creates a new FixedSizeBinaryDictionaryBuilder

pub fn with_capacity( keys_capacity: usize, value_capacity: usize, byte_width: i32, ) -> FixedSizeBinaryDictionaryBuilder<K>

Creates a new FixedSizeBinaryDictionaryBuilder with the provided capacities

keys_capacity: the number of keys, i.e. length of array to build value_capacity: the number of distinct dictionary values, i.e. size of dictionary byte_width: the byte width for individual values in the values array

pub fn try_new_from_builder<K2>( source: FixedSizeBinaryDictionaryBuilder<K2>, ) -> Result<FixedSizeBinaryDictionaryBuilder<K>, ArrowError>

where <K as ArrowPrimitiveType>::Native: NumCast, K2: ArrowDictionaryKeyType, <K2 as ArrowPrimitiveType>::Native: NumCast,

Creates a new FixedSizeBinaryDictionaryBuilder from the existing builder with the same keys and values, but with a new data type for the keys.

Example

let mut u8_keyed_builder = FixedSizeBinaryDictionaryBuilder::<UInt8Type>::new(2);
// appending too many values causes the dictionary to overflow
for i in 0..=255 {
    u8_keyed_builder.append_value(vec![0, i]);
}
let result = u8_keyed_builder.append(vec![1, 0]);
assert!(matches!(result, Err(ArrowError::DictionaryKeyOverflowError{})));

// we need to upgrade to a larger key type
let mut u16_keyed_builder = FixedSizeBinaryDictionaryBuilder::<UInt16Type>::try_new_from_builder(u8_keyed_builder).unwrap();
let dictionary_array = u16_keyed_builder.finish();
let keys = dictionary_array.keys();

assert_eq!(keys, &UInt16Array::from_iter(0..256));

impl<K> FixedSizeBinaryDictionaryBuilder<K>

where K: ArrowDictionaryKeyType,

pub fn append( &mut self, value: impl AsRef<[u8]>, ) -> Result<<K as ArrowPrimitiveType>::Native, ArrowError>

Append a value to the array. Return an existing index if already present in the values array or a new index if the value is appended to the values array.

Returns an error if the new index would overflow the key type.

pub fn append_null(&mut self)

Appends a null slot into the builder

pub fn append_nulls(&mut self, n: usize)

Appends n nulls into the builder.

pub fn append_value(&mut self, value: impl AsRef<[u8]>)

Infallibly append a value to this builder

Panics

Panics if the resulting length of the dictionary values array would exceed T::Native::MAX

pub fn finish(&mut self) -> DictionaryArray<K>

Builds the DictionaryArray and reset this builder.

pub fn finish_cloned(&self) -> DictionaryArray<K>

Builds the DictionaryArray without resetting the builder.

pub fn finish_preserve_values(&mut self) -> DictionaryArray<K>

Builds the DictionaryArray without resetting the values builder or the internal de-duplication map.

The advantage of doing this is that the values will represent the entire set of what has been built so-far by this builder and ensures consistency in the assignment of keys to values across multiple calls to finish_preserve_values. This enables ipc writers to efficiently emit delta dictionaries.

The downside to this is that building the record requires creating a copy of the values, which can become slowly more expensive if the dictionary grows.

Additionally, if record batches from multiple different dictionary builders for the same column are fed into a single ipc writer, beware that entire dictionaries are likely to be re-sent frequently even when the majority of the values are not used by the current record batch.

Trait Implementations

impl<K> ArrayBuilder for FixedSizeBinaryDictionaryBuilder<K>

where K: ArrowDictionaryKeyType,

fn as_any(&self) -> &(dyn Any + 'static)

Returns the builder as an non-mutable Any reference.

fn as_any_mut(&mut self) -> &mut (dyn Any + 'static)

Returns the builder as an mutable Any reference.

fn into_box_any(self: Box<FixedSizeBinaryDictionaryBuilder<K>>) -> Box<dyn Any>

Returns the boxed builder as a box of Any.

fn len(&self) -> usize

Returns the number of array slots in the builder

fn finish(&mut self) -> Arc<dyn Array>

Builds the array and reset this builder.

fn finish_cloned(&self) -> Arc<dyn Array>

Builds the array without resetting the builder.

fn is_empty(&self) -> bool

Returns whether number of array slots is zero

impl<K> Debug for FixedSizeBinaryDictionaryBuilder<K>

where K: Debug + ArrowDictionaryKeyType,

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.