pub enum Canonical {
Null(NullArray),
Bool(BoolArray),
Primitive(PrimitiveArray),
Struct(StructArray),
List(ListArray),
VarBinView(VarBinViewArray),
Extension(ExtensionArray),
}
Expand description
The set of canonical array encodings, also the set of encodings that can be transferred to Arrow with zero-copy.
Note that a canonical form is not recursive, i.e. a StructArray may contain non-canonical child arrays, which may themselves need to be canonicalized.
§Logical vs. Physical encodings
Vortex separates logical and physical types, however this creates ambiguity with Arrow, there is no separation. Thus, if you receive an Arrow array, compress it using Vortex, and then decompress it later to pass to a compute kernel, there are multiple suitable Arrow array variants to hold the data.
To disambiguate, we choose a canonical physical encoding for every Vortex DType
, which
will correspond to an arrow-rs arrow_schema::DataType
.
§Views support
Binary and String views, also known as “German strings” are a better encoding format for
nearly all use-cases. Variable-length binary views are part of the Apache Arrow spec, and are
fully supported by the Datafusion query engine. We use them as our canonical string encoding
for all Utf8
and Binary
typed arrays in Vortex.
Variants§
Null(NullArray)
Bool(BoolArray)
Primitive(PrimitiveArray)
Struct(StructArray)
List(ListArray)
VarBinView(VarBinViewArray)
Extension(ExtensionArray)
Implementations§
Source§impl Canonical
impl Canonical
pub fn into_null(self) -> VortexResult<NullArray>
pub fn into_bool(self) -> VortexResult<BoolArray>
pub fn into_primitive(self) -> VortexResult<PrimitiveArray>
pub fn into_struct(self) -> VortexResult<StructArray>
pub fn into_list(self) -> VortexResult<ListArray>
pub fn into_varbinview(self) -> VortexResult<VarBinViewArray>
pub fn into_extension(self) -> VortexResult<ExtensionArray>
Trait Implementations§
Source§impl From<Canonical> for ArrayRef
This conversion is always “free” and should not touch underlying data. All it does is create an
owned pointer to the underlying concrete array type.
impl From<Canonical> for ArrayRef
This conversion is always “free” and should not touch underlying data. All it does is create an owned pointer to the underlying concrete array type.
This combined with the above ToCanonical impl for ArrayRef allows simple two-way conversions between arbitrary Vortex encodings and canonical Arrow-compatible encodings.
Auto Trait Implementations§
impl !Freeze for Canonical
impl !RefUnwindSafe for Canonical
impl Send for Canonical
impl Sync for Canonical
impl Unpin for Canonical
impl !UnwindSafe for Canonical
Blanket Implementations§
Source§impl<T> ArchivePointee for T
impl<T> ArchivePointee for T
Source§type ArchivedMetadata = ()
type ArchivedMetadata = ()
Source§fn pointer_metadata(
_: &<T as ArchivePointee>::ArchivedMetadata,
) -> <T as Pointee>::Metadata
fn pointer_metadata( _: &<T as ArchivePointee>::ArchivedMetadata, ) -> <T as Pointee>::Metadata
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left
is true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left(&self)
returns true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moreSource§impl<T> LayoutRaw for T
impl<T> LayoutRaw for T
Source§fn layout_raw(_: <T as Pointee>::Metadata) -> Result<Layout, LayoutError>
fn layout_raw(_: <T as Pointee>::Metadata) -> Result<Layout, LayoutError>
Source§impl<T, N1, N2> Niching<NichedOption<T, N1>> for N2
impl<T, N1, N2> Niching<NichedOption<T, N1>> for N2
Source§unsafe fn is_niched(niched: *const NichedOption<T, N1>) -> bool
unsafe fn is_niched(niched: *const NichedOption<T, N1>) -> bool
Source§fn resolve_niched(out: Place<NichedOption<T, N1>>)
fn resolve_niched(out: Place<NichedOption<T, N1>>)
out
indicating that a T
is niched.