pub enum DataType {
}Expand description
An Arrow type.
The subset our own types map onto, which is every type the engine can produce a value of today except the five nested ones.
LIST, STRUCT and MAP are the three of the five that are now only missing here. There has been
a list vector since #302, a struct vector since #594 and a map vector since #595, so for all three of
them the blocker is no longer underneath this crate.
A struct is nearly a transcription. Arrow holds one child array per field, each as long as the
parent, with the parent’s validity bitmap on top, which is exactly Form::Struct’s layout, so the
export is the children exported and a schema saying what they are called. Nothing is rearranged and
nothing is copied that would not have been copied anyway.
A list has a real decision in it. Arrow says where a row ends by saying where the next one begins, and a list vector says where each row starts and how long it is, so a vector whose rows are in order and touching exports as offsets directly and one that has been gathered or filtered has to have its child gathered first. That is the piece to write, and it is a decision about when to pay for the gather rather than a missing layer.
A map costs whatever the list costs and nothing on top of it. Arrow’s map is a list of a two field
struct named entries, which is the layout a map vector already has, so once the list export exists
the map export is the same code with the child’s field names checked.
Variants§
Null
No values, all null, no buffers at all.
Boolean
One bit per value.
Int8
8 bit signed.
Int16
16 bit signed.
Int32
32 bit signed.
Int64
64 bit signed.
UInt8
8 bit unsigned.
UInt16
16 bit unsigned.
UInt32
32 bit unsigned.
UInt64
64 bit unsigned.
Float32
IEEE 754 binary32.
Float64
IEEE 754 binary64.
Utf8
UTF-8, with 32 bit offsets.
Binary
Bytes, with 32 bit offsets.
Date32
Days since 1970-01-01, 32 bit.
Time64
Microseconds since midnight, 64 bit.
Timestamp(TimeUnit, Option<String>)
Microseconds since the epoch, 64 bit, with a time zone when there is one.
Interval
Months, days and nanoseconds, sixteen bytes.
Decimal128
A 128 bit integer with a decimal point in it.
Implementations§
Source§impl DataType
impl DataType
Sourcepub fn of(ty: &LogicalType) -> Result<Self>
pub fn of(ty: &LogicalType) -> Result<Self>
The Arrow type one of ours becomes.
HUGEINT becomes DECIMAL128(38, 0), which is what DuckDB exports it as and the only thing
it can be: Arrow has no 128 bit integer and a decimal with no fractional digits is one.
§Errors
For a type with no Arrow counterpart yet, which is the nested types, UHUGEINT, BIT and
UUID.
Sourcepub fn format(&self) -> String
pub fn format(&self) -> String
The format string the Arrow C data interface names this type with.
Written now, before there is an FFI boundary to hand it across, because it is the part of the mapping that is defined by somebody else’s document and the part a test can check against that document. The export itself is then a struct with this string in it.
Sourcepub fn buffer_count(&self) -> usize
pub fn buffer_count(&self) -> usize
How many buffers an array of this type has, which the C data interface also has to say.
Two for anything fixed width, which is the validity bitmap and the values. Three for a variable width type, which puts the offsets in between. None at all for the null type, which has no values to be valid or invalid.