Enum DType 

pub enum DType {
    Null,
    Bool(Nullability),
    Primitive(PType, Nullability),
    Decimal(DecimalDType, Nullability),
    Utf8(Nullability),
    Binary(Nullability),
    List(Arc<DType>, Nullability),
    FixedSizeList(Arc<DType>, u32, Nullability),
    Struct(StructFields, Nullability),
    Extension(ExtDTypeRef),
}

The logical types of elements in Vortex arrays.

DType represents the different logical data types that can be represented in a Vortex array.

This is different from physical types, which represent the actual layout of data (compressed or uncompressed). The set of physical types/formats (or data layout) is surjective into the set of logical types (or in other words, all physical types map to a single logical type).

Note that a DType represents the logical type of the elements in the Arrays, not the logical type of the Array itself.

For example, an array with DType::Primitive(I32, NonNullable) could be physically encoded as any of the following:

• A flat array of i32 values.
• A run-length encoded sequence.
• Dictionary encoded values with bitpacked codes.

All of these physical encodings preserve the same logical I32 type, even if the physical data is different.

Variants

Null

A logical null type.

Null only has a single value, null.

Bool(Nullability)

A logical boolean type.

Bool can be true or false if non-nullable. It can be true, false, or null if nullable.

Primitive(PType, Nullability)

A logical fixed-width numeric type.

This can be unsigned, signed, or floating point. See PType for more information.

Decimal(DecimalDType, Nullability)

Logical real numbers with fixed precision and scale.

See DecimalDType for more information.

Utf8(Nullability)

Logical UTF-8 strings.

Binary(Nullability)

Logical binary data.

List(Arc<DType>, Nullability)

A logical variable-length list type.

This is parameterized by a single DType that represents the element type of the inner lists.

FixedSizeList(Arc<DType>, u32, Nullability)

A logical fixed-size list type.

This is parameterized by a DType that represents the element type of the inner lists, as well as a u32 size that determines the fixed length of each FixedSizeList scalar.

Struct(StructFields, Nullability)

A logical struct type.

A Struct type is composed of an ordered list of fields, each with a corresponding name and DType. See StructFields for more information.

Extension(ExtDTypeRef)

A user-defined extension type.

See ExtDTypeRef for more information.

Implementations

impl DType

pub fn to_arrow_schema(&self) -> VortexResult<Schema>

Convert a Vortex DType into an Arrow Schema.

pub fn to_arrow_dtype(&self) -> VortexResult<DataType>

Returns the Arrow DataType that best corresponds to this Vortex DType.

impl DType

pub const BYTES: Self

The default DType for bytes.

pub fn nullability(&self) -> Nullability

Get the nullability of the DType.

pub fn is_nullable(&self) -> bool

Check if the DType is Nullability::Nullable.

pub fn as_nonnullable(&self) -> Self

Get a new DType with Nullability::NonNullable (but otherwise the same as self)

pub fn as_nullable(&self) -> Self

Get a new DType with Nullability::Nullable (but otherwise the same as self)

pub fn with_nullability(&self, nullability: Nullability) -> Self

Get a new DType with the given nullability (but otherwise the same as self)

pub fn union_nullability(&self, other: Nullability) -> Self

Union the nullability of this DType with the other nullability, returning a new DType.

pub fn eq_ignore_nullability(&self, other: &Self) -> bool

Check if self and other are equal, ignoring nullability.

pub fn eq_with_nullability_subset(&self, other: &Self) -> bool

Returns true if self is a subset type of other, otherwise false`.

If self is nullable, this means that the other DType must also be nullable (since a nullable type represents more values than a non-nullable type) and equal.

If self is non-nullable, then the other DType must be equal ignoring nullabillity.

We implement this functionality as a complement to is_superset_of.

pub fn eq_with_nullability_superset(&self, other: &Self) -> bool

Returns true if self is a superset type of other, otherwise false`.

If self is non-nullable, this means that the other DType must also be non-nullable (since a non-nullable type represents less values than a nullable type) and equal.

If self is nullable, then the other DType must be equal ignoring nullabillity.

This function is useful (in the vortex-array crate) for determining if an Array can extend a given ArrayBuilder: it can only extend it if the DType of the builder is a superset of the Array.

pub fn is_boolean(&self) -> bool

Check if self is a boolean

pub fn is_primitive(&self) -> bool

Check if self is a primitive type

pub fn as_ptype(&self) -> PType

Returns this DType’s PType if it is a primitive type, otherwise panics.

pub fn is_unsigned_int(&self) -> bool

Check if self is an unsigned integer

pub fn is_signed_int(&self) -> bool

Check if self is a signed integer

pub fn is_int(&self) -> bool

Check if self is an integer (signed or unsigned)

pub fn is_float(&self) -> bool

Check if self is a floating point number

pub fn is_decimal(&self) -> bool

Check if self is a DType::Decimal.

pub fn is_utf8(&self) -> bool

Check if self is a DType::Utf8

pub fn is_binary(&self) -> bool

Check if self is a DType::Binary

pub fn is_list(&self) -> bool

Check if self is a DType::List.

pub fn is_fixed_size_list(&self) -> bool

Check if self is a DType::FixedSizeList,

pub fn is_struct(&self) -> bool

Check if self is a DType::Struct

pub fn is_extension(&self) -> bool

Check if self is a DType::Extension type

pub fn is_nested(&self) -> bool

Check if self is a nested type, i.e. list, fixed size list, struct, or extension of a recursive type.

pub fn element_size(&self) -> Option<usize>

Returns the number of bytes occupied by a single scalar of this fixed-width type.

For non-fixed-width types, return None.

Bool is defined as 1 even though a Vortex array may pack Booleans to one bit per element.

pub fn as_decimal_opt(&self) -> Option<&DecimalDType>

Check returns the inner decimal type if the dtype is a DType::Decimal.

pub fn into_decimal_opt(self) -> Option<DecimalDType>

Owned version of Self::as_decimal_opt.

pub fn as_list_element_opt(&self) -> Option<&Arc<DType>>

Get the inner element dtype if self is a DType::List, otherwise returns None.

Note that this does not return Some if self is a DType::FixedSizeList.

pub fn into_list_element_opt(self) -> Option<Arc<DType>>

Owned version of Self::as_list_element_opt.

pub fn as_fixed_size_list_element_opt(&self) -> Option<&Arc<DType>>

Get the inner element dtype if self is a DType::FixedSizeList, otherwise returns None.

Note that this does not return Some if self is a DType::List.

pub fn into_fixed_size_list_element_opt(self) -> Option<Arc<DType>>

Owned version of Self::as_fixed_size_list_element_opt.

pub fn as_any_size_list_element_opt(&self) -> Option<&Arc<DType>>

Get the inner element dtype if self is either a DType::List or a DType::FixedSizeList, otherwise returns None

pub fn into_any_size_list_element_opt(self) -> Option<Arc<DType>>

Owned version of Self::as_any_size_list_element_opt.

pub fn as_struct_fields(&self) -> &StructFields

Returns the StructFields from a struct DType.

Panics

If the DType is not a struct.

pub fn into_struct_fields(self) -> StructFields

Owned version of Self::as_struct_fields.

pub fn as_struct_fields_opt(&self) -> Option<&StructFields>

Get the StructDType if self is a StructDType, otherwise None

pub fn into_struct_fields_opt(self) -> Option<StructFields>

Owned version of Self::as_struct_fields_opt.

pub fn as_extension(&self) -> &ExtDTypeRef

Downcast a DType to an ExtDType

pub fn as_extension_opt(&self) -> Option<&ExtDTypeRef>

Get the ExtDTypeRef if self is an Extension type, otherwise None

pub fn list(dtype: impl Into<DType>, nullability: Nullability) -> Self

Convenience method for creating a DType::List.

pub fn struct_<I: IntoIterator<Item = (impl Into<FieldName>, impl Into<FieldDType>)>>( iter: I, nullability: Nullability, ) -> Self

Convenience method for creating a DType::Struct.

impl DType

pub fn from_flatbuffer( buffer: FlatBuffer, session: &VortexSession, ) -> VortexResult<Self>

Create a DType from a flatbuffer buffer.

impl DType

pub fn from_proto(value: &DType, session: &VortexSession) -> VortexResult<Self>

Constructs a DType from its protobuf representation.

Trait Implementations

impl<'a> Arbitrary<'a> for DType

fn arbitrary(u: &mut Unstructured<'a>) -> Result<Self>

Generate an arbitrary value of Self from the given unstructured data.

fn arbitrary_take_rest(u: Unstructured<'a>) -> Result<Self, Error>

Generate an arbitrary value of Self from the entirety of the given unstructured data.

fn size_hint(depth: usize) -> (usize, Option<usize>)

Get a size hint for how many bytes out of an Unstructured this type needs to construct itself.

fn try_size_hint( depth: usize, ) -> Result<(usize, Option<usize>), MaxRecursionReached>

Get a size hint for how many bytes out of an Unstructured this type needs to construct itself.

impl Clone for DType

fn clone(&self) -> DType

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for DType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'a> From<&'a DType> for Input<'a>

fn from(value: &'a DType) -> Self

Converts to this type from the input type.

impl From<DType> for FieldDType

fn from(value: DType) -> Self

Converts to this type from the input type.

impl From<PType> for &DType

fn from(item: PType) -> Self

Converts to this type from the input type.

impl From<PType> for DType

fn from(item: PType) -> Self

Converts to this type from the input type.

impl FromArrowType<&Field> for DType

fn from_arrow(field: &Field) -> Self

Convert the Arrow type to a Vortex type.

impl FromArrowType<&Schema> for DType

fn from_arrow(value: &Schema) -> Self

Convert the Arrow type to a Vortex type.

impl FromArrowType<(&DataType, Nullability)> for DType

fn from_arrow((data_type, nullability): (&DataType, Nullability)) -> Self

Convert the Arrow type to a Vortex type.

impl FromArrowType<Arc<Schema>> for DType

fn from_arrow(value: SchemaRef) -> Self

Convert the Arrow type to a Vortex type.

impl Hash for DType

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for DType

fn eq(&self, other: &DType) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for DType

fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>

where S: Serializer,

Serialize this value into the given Serde serializer.

impl TryFrom<&DType> for DType

type Error = VortexError

The type returned in the event of a conversion error.

fn try_from(value: &DType) -> VortexResult<Self>

Performs the conversion.

impl TryFrom<&DType> for DecimalDType

type Error = VortexError

The type returned in the event of a conversion error.

fn try_from(value: &DType) -> Result<Self, Self::Error>

Performs the conversion.

impl TryFrom<&DType> for PType

type Error = VortexError

The type returned in the event of a conversion error.

fn try_from(value: &DType) -> VortexResult<Self>

Performs the conversion.

impl TryFrom<DType> for DecimalDType

type Error = VortexError

The type returned in the event of a conversion error.

fn try_from(value: DType) -> Result<Self, Self::Error>

Performs the conversion.

impl WriteFlatBuffer for DType

type Target<'a> = DType<'a>

The FlatBuffer type that this type can be written to.

fn write_flatbuffer<'fb>( &self, fbb: &mut FlatBufferBuilder<'fb>, ) -> VortexResult<WIPOffset<Self::Target<'fb>>>

Writes this type to a FlatBuffer builder.

impl Eq for DType

impl FlatBufferRoot for DType

impl StructuralPartialEq for DType