Expand description
Vortex crate containing core logic for encoding and memory representation of arrays.
At the heart of Vortex are arrays and encodings. Arrays are typed views of memory buffers that hold scalars. These buffers can be held in a number of physical encodings to perform lightweight compression that exploits the particular data distribution of the array’s values.
Every data type recognized by Vortex also has a canonical physical encoding format, which arrays can be canonicalized into for ease of access in compute functions.
Modules§
- accessor
- aliases
- Re-exports of third-party crates we use in the API.
- arcref
- arrays
- All the built-in encoding schemes and arrays.
- arrow
- Utilities to work with
Arrowdata and types. - builders
- Builders for Vortex arrays.
- compress
- compute
- Compute kernels on top of Vortex Arrays.
- flatbuffers
- Re-exported autogenerated code from the core Vortex flatbuffer definitions.
- iter
- Iterator over slices of an array, and related utilities.
- nbytes
- patches
- serde
- stats
- Traits and utilities to compute and access array statistics.
- stream
- validity
- Array validity and nullability behavior, used by arrays and compute functions.
- variants
- This module defines array traits for each Vortex DType.
- vtable
- This module contains the VTable definitions for a Vortex Array.
Macros§
Structs§
- Empty
Metadata - Empty array metadata
- Rkyv
Metadata - A utility wrapper for automating the serialization of metadata using rkyv.
- Serde
Metadata - A utility wrapper for automating the serialization of metadata using serde into flexbuffers.
- VTable
Context - A collection of encodings that can be addressed by a u16 positional index. This is used to map array encodings and layout encodings when reading from a file.
- VTable
Registry - A registry of encodings that can be used to construct a context for serde.
Enums§
- Canonical
- The set of canonical array encodings, also the set of encodings that can be transferred to Arrow with zero-copy.
Traits§
- Array
- The base trait for all Vortex arrays.
- Array
Buffer Visitor - Array
Canonical Impl - Implementation trait for canonicalization functions.
- Array
Child Visitor - Array
Compute Impl - A trait used to register static kernels for known compute functions.
Dynamic kernels must be returned via the
_find_kernelmethod. - Array
Ext - Array
Impl - A trait used to encapsulate common implementation behaviour for a Vortex
Array. - Array
Statistics - Extension functions for arrays that provide statistics.
- Array
Statistics Impl - Array
Validity Impl - Implementation trait for validity functions.
- Array
Variants - Array
Variants Impl - Implementation trait for downcasting to type-specific traits.
- Array
Visitor - Array
Visitor Ext - Array
Visitor Impl - Deserialize
Metadata - Encoding
- Marker trait for array encodings with their associated Array type.
- Into
Array - Trait for converting a type into a Vortex
ArrayRef. - Serialize
Metadata - ToCanonical
- Trait for types that can be converted from an owned type into an owned array variant.
- TryFrom
Array Ref - Trait for converting a type from a Vortex
ArrayRef, returning an error if the conversion fails. - TryInto
Array - Trait for converting a type into a Vortex
ArrayRef, returning an error if the conversion fails.
Type Aliases§
- Array
Context - A collection of array encodings.
- Array
Ref - A reference counted pointer to a dynamic
Arraytrait object. - Array
Registry - Encoding
Id - EncodingId is a globally unique name of the array’s encoding.