vortex_array::vtable

Trait SerdeVTable

Source 
pub trait SerdeVTable<V: VTable> {
    type Metadata: Debug + SerializeMetadata + DeserializeMetadata;

    // Required methods
    fn metadata(array: &V::Array) -> VortexResult<Option<Self::Metadata>>;
    fn build(
        encoding: &V::Encoding,
        dtype: &DType,
        len: usize,
        metadata: &<Self::Metadata as DeserializeMetadata>::Output,
        buffers: &[ByteBuffer],
        children: &dyn ArrayChildren,
    ) -> VortexResult<V::Array>;
}
Expand description

VTable trait for building an array from its serialized components.

§Guarantees

Required Associated Types§

Source

type Metadata: Debug + SerializeMetadata + DeserializeMetadata

Required Methods§

Source

fn metadata(array: &V::Array) -> VortexResult<Option<Self::Metadata>>

Exports metadata for an array.

All other parts of the array are exported using the crate::vtable::VisitorVTable.

  • If the array does not require serialized metadata, it should return crate::metadata::EmptyMetadata.
  • If the array does not support serialization, it should return None.
Source

fn build( encoding: &V::Encoding, dtype: &DType, len: usize, metadata: &<Self::Metadata as DeserializeMetadata>::Output, buffers: &[ByteBuffer], children: &dyn ArrayChildren, ) -> VortexResult<V::Array>

Build an array from components.

This is called on the file and IPC deserialization pathways, to reconstruct the array from type-erased components.

Encoding implementers should take note that all validation necessary to ensure the encoding is safe to read should happen inside of this method.

§Safety and correctness

This method should never panic, it must always return an error or else it returns a valid Array that meets all the encoding’s preconditions.

For example, the build implementation for a dictionary encoding should ensure that all codes lie in the valid range. For a UTF-8 array, it should check the bytes to ensure they are all valid string data bytes. Any corrupt files or malformed data buffers should be caught here, before returning the deserialized array.

§Validation

Validation is mainly meant to ensure that all internal pointers in the encoding reference valid ranges of data, and that all data conforms to its DType constraints. These ensure that no array operations will panic at runtime, or yield undefined behavior when unsafe operations like get_unchecked use indices in the array buffer.

Examples of the kinds of validation that should be part of the build step:

  • Checking that any offsets buffers point to valid offsets in some other child array
  • Checking that any buffers for data or validity have the appropriate size for the encoding
  • Running UTF-8 validation for any buffers that are expected to hold flat UTF-8 data

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors§