Expand description
Vortex crate containing core logic for encoding and memory representation of arrays.
At the heart of Vortex are arrays and encodings. Arrays are typed views of memory buffers that hold scalars. These buffers can be held in a number of physical encodings to perform lightweight compression that exploits the particular data distribution of the array’s values.
Every data type recognized by Vortex also has a canonical physical encoding format, which arrays can be canonicalized into for ease of access in compute functions.
Re-exports§
pub use paste;
Modules§
- accessor
- aliases
- array
- All the built-in encoding schemes and arrays.
- arrow
- Utilities to work with
Arrowdata and types. - builders
- compress
- compute
- Compute kernels on top of Vortex Arrays.
- encoding
- Traits and types to define shared unique encoding identifiers.
- flatbuffers
- Re-exported autogenerated code from the core Vortex flatbuffer definitions.
- iter
- nbytes
- parts
- patches
- stats
- Traits and utilities to compute and access array statistics.
- stream
- tree
- validate
- validity
- Array validity and nullability behavior, used by arrays and compute functions.
- variants
- This module defines array traits for each Vortex DType.
- visitor
Macros§
- impl_
encoding - Macro to generate all the necessary code for a new type of array encoding. Including:
Structs§
- Array
Children Iterator - A depth-first pre-order iterator over a ArrayData.
- Array
Data - A central type for all Vortex arrays, which are known length sequences of typed and possibly compressed data.
- Children
Collector - Context
- A mapping between an encoding’s ID to an
EncodingRef, used to have a shared view of all available encoding schemes. - Named
Children Collector
Enums§
- Canonical
- The set of canonical array encodings, also the set of encodings that can be transferred to Arrow with zero-copy.
Traits§
- ArrayD
Type - Array
Len - Array
Metadata - Dynamic trait used to represent opaque owned Array metadata
- Array
Trait - Collects together the behavior of an array.
- GetArray
Metadata - Into
Array Data - Consume
selfand turn it into anArrayDatainfallibly. - Into
Array Variant - Trait for types that can be converted from an owned type into an owned array variant.
- Into
Canonical - Support trait for transmuting an array into the canonical encoding for its vortex_dtype::DType.
- Into
CanonicalV Table - Encoding VTable for canonicalizing an array.
- MetadataV
Table - ToArray
Data - TryDeserialize
Array Metadata - TrySerialize
Array Metadata
Type Aliases§
- Context
Ref - An atomic shared reference to a
Context.