Enum DType 

pub enum DType {
    Null,
    Bool(Nullability),
    Primitive(PType, Nullability),
    Decimal(DecimalDType, Nullability),
    Utf8(Nullability),
    Binary(Nullability),
    List(Arc<DType>, Nullability),
    FixedSizeList(Arc<DType>, u32, Nullability),
    Struct(StructFields, Nullability),
    Extension(Arc<ExtDType>),
}

The logical types of elements in Vortex arrays.

DType represents the different logical data types that can be represented in a Vortex array.

This is different from physical types, which represent the actual layout of data (compressed or uncompressed). The set of physical types/formats (or data layout) is surjective into the set of logical types (or in other words, all physical types map to a single logical type).

Note that a DType represents the logical type of the elements in the Arrays, not the logical type of the Array itself.

For example, an array with DType::Primitive(I32, NonNullable) could be physically encoded as any of the following:

• A flat array of i32 values.
• A run-length encoded sequence.
• Dictionary encoded values with bitpacked codes.

All of these physical encodings preserve the same logical I32 type, even if the physical data is different.

Variants

Null

A logical null type.

Null only has a single value, null.

Bool(Nullability)

A logical boolean type.

Bool can be true or false if non-nullable. It can be true, false, or null if nullable.

Primitive(PType, Nullability)

A logical fixed-width numeric type.

This can be unsigned, signed, or floating point. See PType for more information.

Decimal(DecimalDType, Nullability)

Logical real numbers with fixed precision and scale.

See DecimalDType for more information.

Utf8(Nullability)

Logical UTF-8 strings.

Binary(Nullability)

Logical binary data.

List(Arc<DType>, Nullability)

A logical variable-length list type.

This is parameterized by a single DType that represents the element type of the inner lists.

FixedSizeList(Arc<DType>, u32, Nullability)

A logical fixed-size list type.

This is parameterized by a DType that represents the element type of the inner lists, as well as a u32 size that determines the fixed length of each FixedSizeList scalar.

This variant has not yet been implemented. Please add a comment with TODO(connor)[FixedSizeList] if you need to match against DType.

Struct(StructFields, Nullability)

A logical struct type.

A Struct type is composed of an ordered list of fields, each with a corresponding name and DType. See StructFields for more information.

Extension(Arc<ExtDType>)

A user-defined extension type.

See ExtDType for more information.

Implementations

impl DType

pub fn to_arrow_schema(&self) -> VortexResult<Schema>

Convert a Vortex DType into an Arrow Schema.

pub fn to_arrow_dtype(&self) -> VortexResult<DataType>

Returns the Arrow DataType that best corresponds to this Vortex DType.

impl DType

pub const BYTES: Self

The default DType for bytes.

pub fn nullability(&self) -> Nullability

Get the nullability of the DType.

pub fn is_nullable(&self) -> bool

Check if the DType is Nullability::Nullable.

pub fn as_nonnullable(&self) -> Self

Get a new DType with Nullability::NonNullable (but otherwise the same as self)

pub fn as_nullable(&self) -> Self

Get a new DType with Nullability::Nullable (but otherwise the same as self)

pub fn with_nullability(&self, nullability: Nullability) -> Self

Get a new DType with the given nullability (but otherwise the same as self)

pub fn union_nullability(&self, other: Nullability) -> Self

Union the nullability of this DType with the other nullability, returning a new DType.

pub fn eq_ignore_nullability(&self, other: &Self) -> bool

Check if self and other are equal, ignoring nullability.

pub fn eq_with_nullability_subset(&self, other: &Self) -> bool

Returns true if self is a subset type of other, otherwise false`.

If self is nullable, this means that the other DType must also be nullable (since a nullable type represents more values than a non-nullable type) and equal.

If self is non-nullable, then the other DType must be equal ignoring nullabillity.

We implement this functionality as a complement to is_superset_of.

pub fn eq_with_nullability_superset(&self, other: &Self) -> bool

Returns true if self is a superset type of other, otherwise false`.

If self is non-nullable, this means that the other DType must also be non-nullable (since a non-nullable type represents less values than a nullable type) and equal.

If self is nullable, then the other DType must be equal ignoring nullabillity.

This function is useful (in the vortex-array crate) for determining if an Array can extend a given ArrayBuilder: it can only extend it if the DType of the builder is a superset of the Array.

pub fn is_boolean(&self) -> bool

Check if self is a boolean

pub fn is_primitive(&self) -> bool

Check if self is a primitive type

pub fn as_ptype(&self) -> PType

Returns this DType’s PType if it is a primitive type, otherwise panics.

pub fn is_unsigned_int(&self) -> bool

Check if self is an unsigned integer

pub fn is_signed_int(&self) -> bool

Check if self is a signed integer

pub fn is_int(&self) -> bool

Check if self is an integer (signed or unsigned)

pub fn is_float(&self) -> bool

Check if self is a floating point number

pub fn is_decimal(&self) -> bool

Check if self is a DType::Decimal.

pub fn is_utf8(&self) -> bool

Check if self is a DType::Utf8

pub fn is_binary(&self) -> bool

Check if self is a DType::Binary

pub fn is_list(&self) -> bool

Check if self is a DType::List.

pub fn is_fixed_size_list(&self) -> bool

Check if self is a DType::FixedSizeList,

pub fn is_struct(&self) -> bool

Check if self is a DType::Struct

pub fn is_extension(&self) -> bool

Check if self is a DType::Extension type

pub fn as_decimal_opt(&self) -> Option<&DecimalDType>

Check returns the inner decimal type if the dtype is a DType::Decimal.

pub fn as_list_element_opt(&self) -> Option<&Arc<DType>>

Get the inner element dtype if self is a DType::List or DType::FixedSizeList, otherwise returns None

pub fn as_struct_fields_opt(&self) -> Option<&StructFields>

Get the StructDType if self is a StructDType, otherwise None

pub fn list(dtype: impl Into<DType>, nullability: Nullability) -> Self

Convenience method for creating a DType::List.

pub fn struct_<I: IntoIterator<Item = (impl Into<FieldName>, impl Into<FieldDType>)>>( iter: I, nullability: Nullability, ) -> Self

Convenience method for creating a DType::Struct.

impl DType

pub fn try_from_view(fb: DType<'_>, buffer: FlatBuffer) -> VortexResult<Self>

Create a new

Trait Implementations

impl Clone for DType

fn clone(&self) -> DType

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for DType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<&DType> for DType

fn from(value: &DType) -> Self

Converts to this type from the input type.

impl From<DType> for FieldDType

fn from(value: DType) -> Self

Converts to this type from the input type.

impl From<PType> for &DType

fn from(item: PType) -> Self

Converts to this type from the input type.

impl From<PType> for DType

fn from(item: PType) -> Self

Converts to this type from the input type.

impl FromArrowType<&Field> for DType

fn from_arrow(field: &Field) -> Self

Convert the Arrow type to a Vortex type.

impl FromArrowType<&Schema> for DType

fn from_arrow(value: &Schema) -> Self

Convert the Arrow type to a Vortex type.

impl FromArrowType<(&DataType, Nullability)> for DType

fn from_arrow((data_type, nullability): (&DataType, Nullability)) -> Self

Convert the Arrow type to a Vortex type.

impl FromArrowType<Arc<Schema>> for DType

fn from_arrow(value: SchemaRef) -> Self

Convert the Arrow type to a Vortex type.

impl Hash for DType

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for DType

fn eq(&self, other: &DType) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl TryFrom<&DType> for DType

type Error = VortexError

The type returned in the event of a conversion error.

fn try_from(value: &DType) -> Result<Self, Self::Error>

Performs the conversion.

impl TryFrom<&DType> for DecimalDType

type Error = VortexError

The type returned in the event of a conversion error.

fn try_from(value: &DType) -> Result<Self, Self::Error>

Performs the conversion.

impl TryFrom<&DType> for PType

type Error = VortexError

The type returned in the event of a conversion error.

fn try_from(value: &DType) -> VortexResult<Self>

Performs the conversion.

impl TryFrom<DType> for DecimalDType

type Error = VortexError

The type returned in the event of a conversion error.

fn try_from(value: DType) -> Result<Self, Self::Error>

Performs the conversion.

impl WriteFlatBuffer for DType

type Target<'a> = DType<'a>

The FlatBuffer type that this type can be written to.

fn write_flatbuffer<'fb>( &self, fbb: &mut FlatBufferBuilder<'fb>, ) -> WIPOffset<Self::Target<'fb>>

Writes this type to a FlatBuffer builder.

impl Eq for DType

impl FlatBufferRoot for DType

impl StructuralPartialEq for DType