Expand description
Vortex crate containing core logic for encoding and memory representation of arrays.
At the heart of Vortex are arrays and encodings. Arrays are typed views of memory buffers that hold scalars. These buffers can be held in a number of physical encodings to perform lightweight compression that exploits the particular data distribution of the array’s values.
Every data type recognized by Vortex also has a canonical physical encoding format, which arrays can be canonicalized into for ease of access in compute functions.
Modules§
- accessor
- arrays
- All the built-in encoding schemes and arrays.
- arrow
- Utilities to work with
Arrowdata and types. - builders
- Builders for Vortex arrays.
- compute
- Compute kernels on top of Vortex Arrays.
- display
- execution
- expr
- Vortex’s expression language.
- flatbuffers
- Re-exported autogenerated code from the core Vortex flatbuffer definitions.
- iter
- Iterator over slices of an array, and related utilities.
- optimizer
- patches
- pipeline
- search_
sorted - serde
- stats
- Traits and utilities to compute and access array statistics.
- stream
- test_
harness - validity
- Array validity and nullability behavior, used by arrays and compute functions.
- variants
- This module defines extension functionality specific to each Vortex DType.
- vectors
- vtable
- This module contains the VTable definitions for a Vortex encoding.
Macros§
- assert_
arrays_ eq - register_
kernel - Register a kernel for a compute function. See each compute function for the correct type of kernel to register.
- search_
sorted_ conformance_ 9822126946179500710 - Apply #macro_name template to given body
- vtable
Structs§
- Array
Adapter - Adapter struct used to lift the
VTabletrait into an object-safeArrayimplementation. - Array
Session - Empty
Metadata - Empty array metadata
- Encoding
Adapter - Adapter struct used to lift the
VTabletrait into an object-safeEncodingimplementation. - Mask
Future - A future that resolves to a mask.
- Prost
Metadata - A utility wrapper for Prost metadata serialization.
- RawMetadata
- A utility wrapper for raw metadata serialization. This delegates the serialiation step to the arrays’ vtable.
- VTable
Context - A collection of encodings that can be addressed by a u16 positional index. This is used to map array encodings and layout encodings when reading from a file.
Enums§
- Canonical
- An enum capturing the default uncompressed encodings for each Vortex type.
- Precision
- The precision level for structural equality and hashing of arrays.
Traits§
- Array
- The public API trait for all Vortex arrays.
- Array
Buffer Visitor - Array
Child Visitor - ArrayEq
- An equality trait for arrays that represents structural equality with a configurable level of precision. This trait is used primarily to implement common subtree elimination and other array-based caching mechanisms.
- Array
Hash - A hash trait for arrays that represents structural equality with a configurable level of precision. This trait is used primarily to implement common subtree elimination and other array-based caching mechanisms.
- Array
Operator - Array functions as provided by the
OperatorVTable. - Array
Session Ext - Session data for Vortex arrays.
- Array
Visitor - Array
Visitor Ext - Deserialize
Metadata - Trait for deserializing Vortex metadata from a vector of unaligned bytes.
- DynArray
Eq - A dynamic version of
ArrayEq. - DynArray
Hash - A dynamic version of
ArrayHash. - Encoding
- Marker trait for array encodings with their associated Array type.
- Into
Array - Trait for converting a type into a Vortex
ArrayRef. - Serialize
Metadata - Trait for serializing Vortex metadata to a vector of unaligned bytes.
- ToCanonical
- Trait for types that can be converted from an owned type into an owned array variant.
Type Aliases§
- Array
Context - Array
Ref - A reference counted pointer to a dynamic
Arraytrait object. - Array
Registry - Encoding
Id - EncodingId is a globally unique name of the array’s encoding.
- Encoding
Ref