Module datafusion_python::datafusion_common::arrow::array

Statically typed implementations of Arrow Arrays

See arrow_array for examples and usage instructions

Modules

• array
  The concrete array definitions
• builder
  Defines push-based APIs for constructing arrays
• cast
  Defines helper functions for downcasting dyn Array to concrete types
• ffi
  Contains declarations to bind to the C Data Interface.
• ffi_stream
  Contains declarations to bind to the C Stream Interface.
• iterator
  Idiomatic iterators for Array
• run_iterator
  Idiomatic iterator for RunArray
• temporal_conversions
  Conversion methods for dates and times.
• timezone
  Timezone for timestamp arrays
• types
  Zero-sized types used to parameterize generic array implementations

Macros

• downcast_dictionary_array
  Downcast an Array to a DictionaryArray based on its DataType, accepts a number of subsequent patterns to match the data type
• downcast_integer
  Given one or more expressions evaluating to an integer DataType invokes the provided macro m with the corresponding integer ArrowPrimitiveType, followed by any additional arguments
• downcast_primitive
  Given one or more expressions evaluating to primitive DataType invokes the provided macro m with the corresponding ArrowPrimitiveType, followed by any additional arguments
• downcast_primitive_array
  Downcast an Array to a PrimitiveArray based on its DataType accepts a number of subsequent patterns to match the data type
• downcast_run_array
  Downcast an Array to a RunArray based on its DataType, accepts a number of subsequent patterns to match the data type
• downcast_run_end_index
  Given one or more expressions evaluating to an integer DataType invokes the provided macro m with the corresponding integer RunEndIndexType, followed by any additional arguments
• downcast_temporal
  Given one or more expressions evaluating to primitive DataType invokes the provided macro m with the corresponding ArrowPrimitiveType, followed by any additional arguments
• downcast_temporal_array
  Downcast an Array to a temporal PrimitiveArray based on its DataType accepts a number of subsequent patterns to match the data type

Structs

• ArrayData
  A generic representation of Arrow array data which encapsulates common attributes and operations for Arrow array. Specific operations for different arrays types (e.g., primitive, list, struct) are implemented in Array.
• ArrayDataBuilder
  Builder for ArrayData type
• ArrayIter
  An iterator that returns Some(T) or None, that can be used on any ArrayAccessor
• BooleanArray
  An array of boolean values
• BooleanBufferBuilder
  Builder for BooleanBuffer
• BooleanBuilder
  Builder for BooleanArray
• BufferBuilder
  Builder for creating a Buffer object.
• DataTypeLayout
  Layout specification for a data type
• DictionaryArray
  An array of dictionary encoded values
• FixedSizeBinaryArray
  An array of fixed size binary arrays
• FixedSizeBinaryBuilder
  Builder for FixedSizeBinaryArray
• FixedSizeListArray
  An array of [fixed length lists], similar to JSON arrays (e.g. ["A", "B"]).
• FixedSizeListBuilder
  Builder for FixedSizeListArray
• GenericByteArray
  An array of variable length byte arrays
• GenericByteBuilder
  Builder for GenericByteArray
• GenericByteDictionaryBuilder
  Builder for DictionaryArray of GenericByteArray
• GenericByteRunBuilder
  Builder for RunArray of GenericByteArray
• GenericByteViewArray
  Variable-size Binary View Layout: An array of variable length bytes view arrays.
• GenericByteViewBuilder
  A builder for GenericByteViewArray
• GenericListArray
  An array of variable length lists, similar to JSON arrays (e.g. ["A", "B", "C"]).
• GenericListBuilder
  Builder for GenericListArray
• MapArray
  An array of key-value maps
• MapBuilder
  Builder for MapArray
• MapFieldNames
  The Field names for a MapArray
• MutableArrayData
  Struct to efficiently and interactively create an ArrayData from an existing ArrayData by copying chunks.
• NativeAdapter
  An optional primitive value
• NullArray
  An array of null values
• NullBuilder
  Builder for NullArray
• PrimitiveArray
  An array of primitive values, of type ArrowPrimitiveType
• PrimitiveBuilder
  Builder for PrimitiveArray
• PrimitiveDictionaryBuilder
  Builder for DictionaryArray of PrimitiveArray
• PrimitiveRunBuilder
  Builder for RunArray of PrimitiveArray
• RecordBatch
  A two-dimensional batch of column-oriented data with a defined schema.
• RecordBatchIterator
  Generic implementation of RecordBatchReader that wraps an iterator.
• RecordBatchOptions
  Options that control the behaviour used when creating a RecordBatch.
• RunArray
  An array of run-end encoded values
• Scalar
  A wrapper around a single value Array that implements Datum and indicates compute kernels should treat this array as a scalar value (a single value).
• StructArray
  An array of structs
• StructBuilder
  Builder for StructArray
• TypedDictionaryArray
  A DictionaryArray typed on its child values array
• TypedRunArray
  A RunArray typed typed on its child values array
• UnionArray
  An array of values of varying types
• UnionBuilder
  Builder for UnionArray

Enums

• BufferSpec
  Layout specification for a single data type buffer
• Capacities
  Define capacities of child data or data buffers.

Traits

• AnyDictionaryArray
  A DictionaryArray with the key type erased
• Array
  An array in the arrow columnar format
• ArrayAccessor
  A generic trait for accessing the values of an Array
• ArrayBuilder
  Trait for dealing with different array builders at runtime
• ArrowNativeTypeOp
  Trait for ArrowNativeType that adds checked and unchecked arithmetic operations, and totally ordered comparison operations
• ArrowNumericType
  A subtype of primitive type that represents numeric values.
• ArrowPrimitiveType
  Trait for primitive values, bridging the dynamic-typed nature of Arrow (via DataType) with the static-typed nature of rust types (ArrowNativeType) for all types that implement ArrowNativeType.
• AsArray
  An extension trait for dyn Array that provides ergonomic downcasting
• Datum
  A possibly Scalar Array
• OffsetSizeTrait
  A type that can be used within a variable-size array to encode offset information
• RecordBatchReader
  Trait for types that can read RecordBatch’s.
• RecordBatchWriter
  Trait for types that can write RecordBatch’s.

Functions

• as_boolean_array
  Force downcast of an Array, such as an ArrayRef to BooleanArray, panicking on failure.
• as_decimal_arrayDeprecated
  Force downcast of an Array, such as an ArrayRef to Decimal128Array, panic’ing on failure.
• as_dictionary_array
  Force downcast of an Array, such as an ArrayRef to DictionaryArray<T>, panic’ing on failure.
• as_fixed_size_list_array
  Force downcast of an Array, such as an ArrayRef to FixedSizeListArray, panicking on failure.
• as_generic_binary_array
  Force downcast of an Array, such as an ArrayRef to GenericBinaryArray<S>, panicking on failure.
• as_generic_list_array
  Force downcast of an Array, such as an ArrayRef to GenericListArray<T>, panicking on failure.
• as_large_list_array
  Force downcast of an Array, such as an ArrayRef to LargeListArray, panicking on failure.
• as_largestring_array
  Force downcast of an Array, such as an ArrayRef to LargeStringArray, panicking on failure.
• as_list_array
  Force downcast of an Array, such as an ArrayRef to ListArray, panicking on failure.
• as_map_array
  Force downcast of an Array, such as an ArrayRef to MapArray, panicking on failure.
• as_null_array
  Force downcast of an Array, such as an ArrayRef to NullArray, panicking on failure.
• as_primitive_array
  Force downcast of an Array, such as an ArrayRef, to PrimitiveArray<T>, panic’ing on failure.
• as_run_array
  Force downcast of an Array, such as an ArrayRef to RunArray<T>, panic’ing on failure.
• as_string_array
  Force downcast of an Array, such as an ArrayRef to StringArray, panicking on failure.
• as_struct_array
  Force downcast of an Array, such as an ArrayRef to StructArray, panicking on failure.
• as_union_array
  Force downcast of an Array, such as an ArrayRef to UnionArray, panicking on failure.
• downcast_array
  Downcasts a dyn Array to a concrete type
• export_array_into_raw^⚠Deprecated
  Exports an array to raw pointers of the C Data Interface provided by the consumer.
• layout
  Return the expected DataTypeLayout Arrays of this data type are expected to have
• make_array
  Constructs an array using the input data. Returns a reference-counted Array instance.
• make_builder
  Returns a builder with capacity capacity that corresponds to the datatype DataType This function is useful to construct arrays from an arbitrary vectors with known/expected schema.
• make_comparator
  Returns a comparison function that compares two values at two different positions between the two arrays.
• make_view
  Create a view based on the given data, block id and offset Note that the code below is carefully examined with x86_64 assembly code: https://godbolt.org/z/685YPsd5G The goal is to avoid calling into ptr::copy_non_interleave, which makes function call (i.e., not inlined), which slows down things.
• new_empty_array
  Creates a new empty array
• new_null_array
  Creates a new array of data_type of length length filled entirely of NULL values

Type Aliases

• ArrayDataRef
• ArrayRef
  A reference-counted reference to a generic Array
• BinaryArray
  A GenericBinaryArray of [u8] using i32 offsets
• BinaryBuilder
  Builder for BinaryArray
• BinaryDictionaryBuilder
  Builder for DictionaryArray of BinaryArray
• BinaryRunBuilder
  Builder for RunArray of BinaryArray
• BinaryViewArray
  A GenericByteViewArray of [u8]
• BinaryViewBuilder
  Array builder for BinaryViewArray
• BooleanIter
  an iterator that returns Some(T) or None, that can be used on any BooleanArray
• Date32Array
  A PrimitiveArray of days since UNIX epoch stored as i32
• Date32BufferBuilder
  Buffer builder for 32-bit date type.
• Date32Builder
  A 32-bit date array builder.
• Date64Array
  A PrimitiveArray of milliseconds since UNIX epoch stored as i64
• Date64BufferBuilder
  Buffer builder for 64-bit date type.
• Date64Builder
  A 64-bit date array builder.
• Decimal128Array
  A PrimitiveArray of 128-bit fixed point decimals
• Decimal128BufferBuilder
  Buffer builder for 128-bit decimal type.
• Decimal128Builder
  A decimal 128 array builder
• Decimal256Array
  A PrimitiveArray of 256-bit fixed point decimals
• Decimal256BufferBuilder
  Buffer builder for 256-bit decimal type.
• Decimal256Builder
  A decimal 256 array builder
• DurationMicrosecondArray
  A PrimitiveArray of elapsed durations in microseconds
• DurationMicrosecondBufferBuilder
  Buffer builder for elaspsed time of microseconds unit.
• DurationMicrosecondBuilder
  An elapsed time in microseconds array builder.
• DurationMillisecondArray
  A PrimitiveArray of elapsed durations in milliseconds
• DurationMillisecondBufferBuilder
  Buffer builder for elaspsed time of milliseconds unit.
• DurationMillisecondBuilder
  An elapsed time in milliseconds array builder.
• DurationNanosecondArray
  A PrimitiveArray of elapsed durations in nanoseconds
• DurationNanosecondBufferBuilder
  Buffer builder for elaspsed time of nanoseconds unit.
• DurationNanosecondBuilder
  An elapsed time in nanoseconds array builder.
• DurationSecondArray
  A PrimitiveArray of elapsed durations in seconds
• DurationSecondBufferBuilder
  Buffer builder for elaspsed time of second unit.
• DurationSecondBuilder
  An elapsed time in seconds array builder.
• DynComparator
  Compare the values at two arbitrary indices in two arrays.
• FixedSizeBinaryIter
  an iterator that returns Some(T) or None, that can be used on any FixedSizeBinaryArray
• FixedSizeListIter
  an iterator that returns Some(T) or None, that can be used on any FixedSizeListArray
• Float16Array
  A PrimitiveArray of f16
• Float16BufferBuilder
  Buffer builder for 16-bit floating point type.
• Float16Builder
  A 16-bit floating point array builder.
• Float32Array
  A PrimitiveArray of f32
• Float32BufferBuilder
  Buffer builder for 32-bit floating point type.
• Float32Builder
  A 32-bit floating point array builder.
• Float64Array
  A PrimitiveArray of f64
• Float64BufferBuilder
  Buffer builder for 64-bit floating point type.
• Float64Builder
  A 64-bit floating point array builder.
• GenericBinaryArray
  A GenericBinaryArray for storing [u8]
• GenericBinaryBuilder
  Array builder for GenericBinaryArray
• GenericBinaryIter
  an iterator that returns Some(T) or None, that can be used on any BinaryArray
• GenericListArrayIter
  an iterator that returns Some(T) or None, that can be used on any ListArray
• GenericStringArray
  A GenericByteArray for storing str
• GenericStringBuilder
  Array builder for GenericStringArray
• GenericStringIter
  an iterator that returns Some(T) or None, that can be used on any Utf8Array
• Int8Array
  A PrimitiveArray of i8
• Int8BufferBuilder
  Buffer builder for signed 8-bit integer type.
• Int8Builder
  A signed 8-bit integer array builder.
• Int8DictionaryArray
  A DictionaryArray indexed by i8
• Int16Array
  A PrimitiveArray of i16
• Int16BufferBuilder
  Buffer builder for signed 16-bit integer type.
• Int16Builder
  A signed 16-bit integer array builder.
• Int16DictionaryArray
  A DictionaryArray indexed by i16
• Int16RunArray
  A RunArray with i16 run ends
• Int32Array
  A PrimitiveArray of i32
• Int32BufferBuilder
  Buffer builder for signed 32-bit integer type.
• Int32Builder
  A signed 32-bit integer array builder.
• Int32DictionaryArray
  A DictionaryArray indexed by i32
• Int32RunArray
  A RunArray with i32 run ends
• Int64Array
  A PrimitiveArray of i64
• Int64BufferBuilder
  Buffer builder for signed 64-bit integer type.
• Int64Builder
  A signed 64-bit integer array builder.
• Int64DictionaryArray
  A DictionaryArray indexed by i64
• Int64RunArray
  A RunArray with i64 run ends
• IntervalDayTimeArray
  A PrimitiveArray of “calendar” intervals in days and milliseconds
• IntervalDayTimeBufferBuilder
  Buffer builder for “calendar” interval in days and milliseconds.
• IntervalDayTimeBuilder
  A “calendar” interval in days and milliseconds array builder.
• IntervalMonthDayNanoArray
  A PrimitiveArray of “calendar” intervals in months, days, and nanoseconds.
• IntervalMonthDayNanoBufferBuilder
  Buffer builder “calendar” interval in months, days, and nanoseconds.
• IntervalMonthDayNanoBuilder
  A “calendar” interval in months, days, and nanoseconds array builder.
• IntervalYearMonthArray
  A PrimitiveArray of “calendar” intervals in whole months
• IntervalYearMonthBufferBuilder
  Buffer builder for “calendar” interval in months.
• IntervalYearMonthBuilder
  A “calendar” interval in months array builder.
• LargeBinaryArray
  A GenericBinaryArray of [u8] using i64 offsets
• LargeBinaryBuilder
  Builder for LargeBinaryArray
• LargeBinaryDictionaryBuilder
  Builder for DictionaryArray of LargeBinaryArray
• LargeBinaryRunBuilder
  Builder for RunArray of LargeBinaryArray
• LargeListArray
  A GenericListArray of variable size lists, storing offsets as i64.
• LargeListBuilder
  Builder for LargeListArray
• LargeStringArray
  A GenericStringArray of str using i64 offsets
• LargeStringBuilder
  Builder for LargeStringArray
• LargeStringDictionaryBuilder
  Builder for DictionaryArray of LargeStringArray
• LargeStringRunBuilder
  Builder for RunArray of LargeStringArray
• ListArray
  A GenericListArray of variable size lists, storing offsets as i32.
• ListBuilder
  Builder for ListArray
• MapArrayIter
  an iterator that returns Some(T) or None, that can be used on any MapArray
• PrimitiveIter
  an iterator that returns Some(T) or None, that can be used on any PrimitiveArray
• StringArray
  A GenericStringArray of str using i32 offsets
• StringBuilder
  Builder for StringArray
• StringDictionaryBuilder
  Builder for DictionaryArray of StringArray
• StringRunBuilder
  Builder for RunArray of StringArray
• StringViewArray
  A GenericByteViewArray that stores utf8 data
• StringViewBuilder
  Array builder for StringViewArray
• Time32MillisecondArray
  A PrimitiveArray of milliseconds since midnight stored as i32
• Time32MillisecondBufferBuilder
  Buffer builder for 32-bit elaspsed time since midnight of millisecond unit.
• Time32MillisecondBuilder
  A 32-bit elaspsed time in milliseconds array builder.
• Time32SecondArray
  A PrimitiveArray of seconds since midnight stored as i32
• Time32SecondBufferBuilder
  Buffer builder for 32-bit elaspsed time since midnight of second unit.
• Time32SecondBuilder
  A 32-bit elaspsed time in seconds array builder.
• Time64MicrosecondArray
  A PrimitiveArray of microseconds since midnight stored as i64
• Time64MicrosecondBufferBuilder
  Buffer builder for 64-bit elaspsed time since midnight of microsecond unit.
• Time64MicrosecondBuilder
  A 64-bit elaspsed time in microseconds array builder.
• Time64NanosecondArray
  A PrimitiveArray of nanoseconds since midnight stored as i64
• Time64NanosecondBufferBuilder
  Buffer builder for 64-bit elaspsed time since midnight of nanosecond unit.
• Time64NanosecondBuilder
  A 64-bit elaspsed time in nanoseconds array builder.
• TimestampMicrosecondArray
  A PrimitiveArray of microseconds since UNIX epoch stored as i64
• TimestampMicrosecondBufferBuilder
  Buffer builder for timestamp type of microsecond unit.
• TimestampMicrosecondBuilder
  A timestamp microsecond array builder.
• TimestampMillisecondArray
  A PrimitiveArray of milliseconds since UNIX epoch stored as i64
• TimestampMillisecondBufferBuilder
  Buffer builder for timestamp type of millisecond unit.
• TimestampMillisecondBuilder
  A timestamp millisecond array builder.
• TimestampNanosecondArray
  A PrimitiveArray of nanoseconds since UNIX epoch stored as i64
• TimestampNanosecondBufferBuilder
  Buffer builder for timestamp type of nanosecond unit.
• TimestampNanosecondBuilder
  A timestamp nanosecond array builder.
• TimestampSecondArray
  A PrimitiveArray of seconds since UNIX epoch stored as i64
• TimestampSecondBufferBuilder
  Buffer builder for timestamp type of second unit.
• TimestampSecondBuilder
  A timestamp second array builder.
• UInt8Array
  A PrimitiveArray of u8
• UInt8BufferBuilder
  Buffer builder for usigned 8-bit integer type.
• UInt8Builder
  An usigned 8-bit integer array builder.
• UInt8DictionaryArray
  A DictionaryArray indexed by u8
• UInt16Array
  A PrimitiveArray of u16
• UInt16BufferBuilder
  Buffer builder for usigned 16-bit integer type.
• UInt16Builder
  An usigned 16-bit integer array builder.
• UInt16DictionaryArray
  A DictionaryArray indexed by u16
• UInt32Array
  A PrimitiveArray of u32
• UInt32BufferBuilder
  Buffer builder for usigned 32-bit integer type.
• UInt32Builder
  An usigned 32-bit integer array builder.
• UInt32DictionaryArray
  A DictionaryArray indexed by u32
• UInt64Array
  A PrimitiveArray of u64
• UInt64BufferBuilder
  Buffer builder for usigned 64-bit integer type.
• UInt64Builder
  An usigned 64-bit integer array builder.
• UInt64DictionaryArray
  A DictionaryArray indexed by u64