Module dtype 

A type system for Vortex.

This crate contains the core logical type system for Vortex, including the definition of data types, and (optionally) logic for their serialization and deserialization.

Vortex dtypes are logical domains, not physical layouts. Encodings are tracked separately on arrays, so the same dtype may be backed by canonical buffers, dictionary codes, compressed children, or a lazy expression array.

Every non-null logical dtype carries Nullability directly. This differs from Apache Arrow, where nullability is usually field metadata.

Re-exports

pub use half;

Modules

arrow

Convert between Vortex crate::dtype::DType and Apache Arrow arrow_schema::DataType.

extension

Extension DTypes, and interfaces for working with extension types.

flatbuffers

Flatbuffer representations for DTypes

proto

Protocol buffer representations for DTypes

serde

Serde serialization and deserialization for DTypes

session

Module for managing extension dtypes in a Vortex session.

Macros

field_path

A helpful constructor for creating FieldPaths to nested struct fields of the format field_path!(x.y.z)

match_each_float_ptype

Macro to match over each floating point type, binding the corresponding native type (from NativePType)

match_each_integer_ptype

Macro to match over each integer PType, binding the corresponding native type (from NativePType)

match_each_native_ptype

Macro to match over each PType, binding the corresponding native type (from NativePType)

match_each_native_simd_ptype

Macro to match over each SIMD capable PType, binding the corresponding native type (from NativePType)

match_each_signed_integer_ptype

Macro to match over each signed integer type, binding the corresponding native type (from NativePType)

match_each_unsigned_integer_ptype

Macro to match over each unsigned integer type, binding the corresponding native type (from NativePType)

match_smallest_offset_type

Macro to match the smallest offset type for a given value

Structs

DecimalDType

Parameters that define the precision and scale of a decimal type.

FieldDType

DType of a struct’s field, either owned or a pointer to an underlying flatbuffer.

FieldName

A name for a field in a struct.

FieldNames

An ordered list of field names in a struct.

FieldNamesIntoIter

Owned iterator of field names.

FieldNamesIter

Iterator of references to field names.

FieldPath

A sequence of field selectors representing a path through zero or more layers of DType.

FieldPathSet

A set of field paths supporting efficient contains queries.

PrecisionScale

A struct representing the precision and scale of a decimal type, to be represented by the native type D.

StructFields

Type information for a struct column.

i256

Signed 256-bit integer type.

Enums

DType

The logical types of elements in Vortex arrays.

DecimalType

Type of the decimal values.

Field

Selects a nested type within either a struct or a list.

FieldMask

A projection of fields under a layout.

Nullability

Whether an instance of a DType can be `null or not

PType

Physical type enum, represents the in-memory physical layout but might represent a different logical type.

Constants

MAX_PRECISION

The maximum precision allowed for a decimal type.

MAX_SCALE

The maximum scale allowed for a decimal type.

Traits

BigCast

Checked numeric casts up to and including i256 support.

DecimalTypeDowncast

Trait for downcasting decimal values to native integer types.

DecimalTypeUpcast

Trait for upcasting native integer types to decimal values.

FromPrimitiveOrF16

A trait for types that can be created from primitive values, including f16.

IntegerPType

Trait for integer primitive types that can be used as indices, offsets, or codes.

NativeDType

This trait is implemented by native Rust types that can be converted to and from Vortex scalar values. e.g. &str -> DType::Utf8 bool -> DType::Bool

NativeDecimalType

Type of decimal scalar values.

NativePType

A trait for native Rust types that correspond 1:1 to a PType.

PTypeDowncast

A visitor trait for converting a NativePType to another parameterized type.

PTypeDowncastExt

Extension trait to provide generic downcasting for PTypeDowncast.

PTypeUpcast

A visitor trait for converting a generic NativePType into a non-parameterized type.

PhysicalPType

A trait that allows conversion from a PType to its physical representation (i.e., unsigned)

ToBytes

A trait for types that can be converted to a little-endian byte slice

ToI256

Types that can potentially be converted to an i256.

TryFromBytes

A trait for types that can be converted from a little-endian byte slice

UnsignedPType

Trait for unsigned integer primitive types used where non-negative values are required.