pub trait VTable:
'static
+ Sized
+ Send
+ Sync
+ Debug {
type Array: 'static + Send + Sync + Clone + Debug + Deref<Target = dyn Array> + IntoArray;
type Metadata: Debug;
type ArrayVTable: BaseArrayVTable<Self>;
type CanonicalVTable: CanonicalVTable<Self>;
type OperationsVTable: OperationsVTable<Self>;
type ValidityVTable: ValidityVTable<Self>;
type VisitorVTable: VisitorVTable<Self>;
type ComputeVTable: ComputeVTable<Self>;
type EncodeVTable: EncodeVTable<Self>;
// Required methods
fn id(&self) -> ArrayId;
fn encoding(array: &Self::Array) -> ArrayVTable;
fn metadata(array: &Self::Array) -> VortexResult<Self::Metadata>;
fn serialize(metadata: Self::Metadata) -> VortexResult<Option<Vec<u8>>>;
fn deserialize(bytes: &[u8]) -> VortexResult<Self::Metadata>;
fn build(
&self,
dtype: &DType,
len: usize,
metadata: &Self::Metadata,
buffers: &[BufferHandle],
children: &dyn ArrayChildren,
) -> VortexResult<Self::Array>;
fn with_children(
array: &mut Self::Array,
children: Vec<ArrayRef>,
) -> VortexResult<()>;
// Provided methods
fn execute(
array: &Self::Array,
ctx: &mut ExecutionCtx,
) -> VortexResult<Vector> { ... }
fn execute_parent(
array: &Self::Array,
parent: &ArrayRef,
child_idx: usize,
ctx: &mut ExecutionCtx,
) -> VortexResult<Option<Vector>> { ... }
fn reduce(array: &Self::Array) -> VortexResult<Option<ArrayRef>> { ... }
fn reduce_parent(
array: &Self::Array,
parent: &ArrayRef,
child_idx: usize,
) -> VortexResult<Option<ArrayRef>> { ... }
}Expand description
The array VTable encapsulates logic for an Array type within Vortex.
The logic is split across several “VTable” traits to enable easier code organization than simply lumping everything into a single trait.
Some of these vtables are optional, such as the ComputeVTable and EncodeVTable,
which can be disabled by assigning to the NotSupported type.
From this VTable trait, we derive implementations for the sealed Array and DynVTable
traits via the crate::ArrayAdapter and ArrayVTableAdapter types respectively.
The functions defined in these vtable traits will typically document their pre- and
post-conditions. The pre-conditions are validated inside the Array and DynVTable
implementations so do not need to be checked in the vtable implementations (for example, index
out of bounds). Post-conditions are validated after invocation of the vtable function and will
panic if violated.
Required Associated Types§
type Array: 'static + Send + Sync + Clone + Debug + Deref<Target = dyn Array> + IntoArray
type Metadata: Debug
type ArrayVTable: BaseArrayVTable<Self>
type CanonicalVTable: CanonicalVTable<Self>
type OperationsVTable: OperationsVTable<Self>
type ValidityVTable: ValidityVTable<Self>
type VisitorVTable: VisitorVTable<Self>
Sourcetype ComputeVTable: ComputeVTable<Self>
type ComputeVTable: ComputeVTable<Self>
Optionally enable implementing dynamic compute dispatch for this encoding.
Can be disabled by assigning to the NotSupported type.
Sourcetype EncodeVTable: EncodeVTable<Self>
type EncodeVTable: EncodeVTable<Self>
Optionally enable the EncodeVTable for this encoding. This allows it to partake in
compression.
Can be disabled by assigning to the NotSupported type.
Required Methods§
Sourcefn encoding(array: &Self::Array) -> ArrayVTable
fn encoding(array: &Self::Array) -> ArrayVTable
Returns the encoding for the array.
Sourcefn metadata(array: &Self::Array) -> VortexResult<Self::Metadata>
fn metadata(array: &Self::Array) -> VortexResult<Self::Metadata>
Exports metadata for an array.
All other parts of the array are exported using the crate::vtable::VisitorVTable.
- If the array does not contain metadata, it should return
crate::metadata::EmptyMetadata.
Sourcefn serialize(metadata: Self::Metadata) -> VortexResult<Option<Vec<u8>>>
fn serialize(metadata: Self::Metadata) -> VortexResult<Option<Vec<u8>>>
Serialize metadata into a byte buffer for IPC or file storage.
Return None if the array cannot be serialized.
Sourcefn deserialize(bytes: &[u8]) -> VortexResult<Self::Metadata>
fn deserialize(bytes: &[u8]) -> VortexResult<Self::Metadata>
Deserialize metadata from a byte buffer.
Sourcefn build(
&self,
dtype: &DType,
len: usize,
metadata: &Self::Metadata,
buffers: &[BufferHandle],
children: &dyn ArrayChildren,
) -> VortexResult<Self::Array>
fn build( &self, dtype: &DType, len: usize, metadata: &Self::Metadata, buffers: &[BufferHandle], children: &dyn ArrayChildren, ) -> VortexResult<Self::Array>
Build an array from components.
This is called on the file and IPC deserialization pathways, to reconstruct the array from type-erased components.
Encoding implementers should take note that all validation necessary to ensure the encoding is safe to read should happen inside of this method.
§Safety and correctness
This method should never panic, it must always return an error or else it returns a
valid Array that meets all the encoding’s preconditions.
For example, the build implementation for a dictionary encoding should ensure that all
codes lie in the valid range. For a UTF-8 array, it should check the bytes to ensure they
are all valid string data bytes. Any corrupt files or malformed data buffers should be
caught here, before returning the deserialized array.
§Validation
Validation is mainly meant to ensure that all internal pointers in the encoding reference
valid ranges of data, and that all data conforms to its DType constraints. These ensure
that no array operations will panic at runtime, or yield undefined behavior when unsafe
operations like get_unchecked use indices in the array buffer.
Examples of the kinds of validation that should be part of the build step:
- Checking that any offsets buffers point to valid offsets in some other child array
- Checking that any buffers for data or validity have the appropriate size for the encoding
- Running UTF-8 validation for any buffers that are expected to hold flat UTF-8 data
Sourcefn with_children(
array: &mut Self::Array,
children: Vec<ArrayRef>,
) -> VortexResult<()>
fn with_children( array: &mut Self::Array, children: Vec<ArrayRef>, ) -> VortexResult<()>
Replaces the children in array with children. The count must be the same and types
of children must be expected.
Provided Methods§
Sourcefn execute(array: &Self::Array, ctx: &mut ExecutionCtx) -> VortexResult<Vector>
fn execute(array: &Self::Array, ctx: &mut ExecutionCtx) -> VortexResult<Vector>
Execute this array to produce a Vector.
The returned Vector must be the appropriate one for the array’s logical
type (they are one-to-one with Vortex DTypes), and should respect the output nullability
of the array.
Debug builds will panic if the returned vector is of the wrong type, wrong length, or incorrectly contains null values.
Implementations should recursively call crate::executor::VectorExecutor::execute on
child arrays as needed.
Sourcefn execute_parent(
array: &Self::Array,
parent: &ArrayRef,
child_idx: usize,
ctx: &mut ExecutionCtx,
) -> VortexResult<Option<Vector>>
fn execute_parent( array: &Self::Array, parent: &ArrayRef, child_idx: usize, ctx: &mut ExecutionCtx, ) -> VortexResult<Option<Vector>>
Attempt to execute the parent of this array to produce a Vector.
This function allows arrays to plug in specialized execution logic for their parent. For example, strings compressed as FSST arrays can implement a custom equality comparison when the comparing against a scalar string.
Returns Ok(None) if no specialized execution is possible.
Sourcefn reduce(array: &Self::Array) -> VortexResult<Option<ArrayRef>>
fn reduce(array: &Self::Array) -> VortexResult<Option<ArrayRef>>
Attempt to reduce the array to a more simple representation.
Returns Ok(None) if no reduction is possible.
Sourcefn reduce_parent(
array: &Self::Array,
parent: &ArrayRef,
child_idx: usize,
) -> VortexResult<Option<ArrayRef>>
fn reduce_parent( array: &Self::Array, parent: &ArrayRef, child_idx: usize, ) -> VortexResult<Option<ArrayRef>>
Attempt to perform a reduction of the parent of this array.
This function allows arrays to plug in reduction rules to their parents, for example run-end arrays can pull-down scalar functions and apply them only over their values.
Returns Ok(None) if no reduction is possible.
Dyn Compatibility§
This trait is not dyn compatible.
In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.