Expand description
Statically typed implementations of Arrow Arrays
See arrow_array for examples and usage instructions
Structs
- An generic representation of Arrow array data which encapsulates common attributes and operations for Arrow array. Specific operations for different arrays types (e.g., primitive, list, struct) are implemented in
Array
. - Builder for
ArrayData
type - An iterator that returns Some(T) or None, that can be used on any
ArrayAccessor
- Array of bools
- A builder for creating a boolean
Buffer
- Array builder for fixed-width primitive types
- Builder for creating a
Buffer
object. - Layout specification for a data type
- A dictionary array where each element is a single value indexed by an integer key. This is mostly used to represent strings or a limited set of primitive types as integers, for example when doing NLP analysis or representing chromosomes by name.
- An array where each element is a fixed-size sequence of bytes.
- A fixed size binary array builder
- A list array where each element is a fixed-size sequence of values with the same type whose maximum length is represented by a i32.
- Array builder for
FixedSizeListArray
- Generic struct for variable-size byte arrays
- Array builder for
GenericByteArray
- Generic array builder for
DictionaryArray
that stores generic byte values. For example to map a set of byte indices to String values. Note that the use of aHashMap
here will not scale to very large arrays or result in an ordered dictionary. - Array builder for
RunArray
for String and Binary types. - Generic struct for a variable-size list array.
- Array builder for
GenericListArray
s. - A nested array type where each record is a key-value map. Keys should always be non-null, but values can be null.
- Creates a new
MapBuilder
- Contains details of the mapping
- This struct is used as an adapter when creating
PrimitiveArray
from an iterator.FromIterator
forPrimitiveArray
takes an iterator where the elements can beinto
this struct. So once implementingFrom
orInto
trait for a type, an iterator of the type can be collected toPrimitiveArray
. - An Array where all elements are nulls
- Array whose elements are of primitive types.
- Array builder for fixed-width primitive types
- Array builder for
DictionaryArray
. For example to map a set of byte indices to f32 values. Note that the use of aHashMap
here will not scale to very large arrays or result in an ordered dictionary. - Array builder for
RunArray
that encodes primitive values. - A run-end encoding (REE) is a variation of run-length encoding (RLE).
- A nested array type where each child (called field) is represented by a separate array.
- Array builder for Struct types.
- A strongly-typed wrapper around a
DictionaryArray
that implementsArrayAccessor
allowing fast access to its elements - A strongly-typed wrapper around a
RunArray
that implementsArrayAccessor
andIntoIterator
allowing fast access to its elements - An Array that can represent slots of varying types.
- Builder type for creating a new
UnionArray
.
Enums
- Layout specification for a single data type buffer
- Define capacities of child data or data buffers.
Traits
- Trait for dealing with different types of array at runtime when the type of the array is not known in advance.
- A generic trait for accessing the values of an
Array
- Trait for dealing with different array builders at runtime
- Trait bridging the dynamic-typed nature of Arrow (via
DataType
) with the static-typed nature of rust types (ArrowNativeType
) for all types that implementArrowNativeType
. - An extension trait for
dyn Array
that provides ergonomic downcasting - trait declaring an offset size, relevant for i32 vs i64 array types.
Functions
- returns a comparison function that compares two values at two different positions between the two arrays. The arrays’ types must be equal.
- Downcasts a
dyn Array
to a concrete type - export_array_into_raw⚠DeprecatedExports an array to raw pointers of the C Data Interface provided by the consumer.
- Return the expected
DataTypeLayout
Arrays of this data type are expected to have - Constructs an array using the input
data
. Returns a reference-countedArray
instance. - Returns a builder with capacity
capacity
that corresponds to the datatypeDataType
This function is useful to construct arrays from an arbitrary vectors with known/expected schema. - Creates a new empty array
- Creates a new array of
data_type
of lengthlength
filled entirely ofNULL
values
Type Definitions
- A reference-counted reference to a generic
Array
. - An array where each element contains 0 or more bytes. The byte length of each element is represented by an i32.
- A binary array builder with i32 offsets
- Array builder for
DictionaryArray
that stores binary. For example to map a set of byte indices to binary values. Note that the use of aHashMap
here will not scale to very large arrays or result in an ordered dictionary. - Array builder for
RunArray
that encodes binary values(BinaryType
). - an iterator that returns Some(T) or None, that can be used on any BooleanArray
- A primitive array where each element is of 32-bit value representing the elapsed time since UNIX epoch in days.“
- Buffer builder for 32-bit date type.
- A 32-bit date array builder.
- A primitive array where each element is a 64-bit value representing the elapsed time since the UNIX epoch in milliseconds.
- Buffer builder for 64-bit date type.
- A 64-bit date array builder.
- An array where each element is a 128-bits decimal with precision in [1, 38] and scale less or equal to 38.
- A decimal 128 array builder
- An array where each element is a 256-bits decimal with precision in [1, 76] and scale less or equal to 76.
- A decimal 256 array builder
- An array where each element is an elapsed time type in microseconds.
- Buffer builder for elaspsed time of microseconds unit.
- An elapsed time in microseconds array builder.
- An array where each element is an elapsed time type in milliseconds.
- Buffer builder for elaspsed time of milliseconds unit.
- An elapsed time in milliseconds array builder.
- An array where each element is an elapsed time type in nanoseconds.
- Buffer builder for elaspsed time of nanoseconds unit.
- An elapsed time in nanoseconds array builder.
- An array where each element is an elapsed time type in seconds.
- Buffer builder for elaspsed time of second unit.
- An elapsed time in seconds array builder.
- Compare the values at two arbitrary indices in two arrays.
- an iterator that returns Some(T) or None, that can be used on any FixedSizeBinaryArray
- Example: Using
collect
- Example: Using
collect
- Buffer builder for 32-bit floating point type.
- A 32-bit floating point array builder.
- Example: Using
collect
- Buffer builder for 64-bit floating point type.
- A 64-bit floating point array builder.
- See
BinaryArray
andLargeBinaryArray
for storing binary data. - Array builder for
GenericBinaryArray
- an iterator that returns Some(T) or None, that can be used on any BinaryArray
- an iterator that returns Some(T) or None, that can be used on any ListArray
- Generic struct for [Large]StringArray
- Array builder for
GenericStringArray
- an iterator that returns Some(T) or None, that can be used on any Utf8Array
- Example: Using
collect
- Buffer builder for signed 8-bit integer type.
- A signed 8-bit integer array builder.
- A dictionary array where each element is a single value indexed by an integer key.
- Example: Using
collect
- Buffer builder for signed 16-bit integer type.
- A signed 16-bit integer array builder.
- A dictionary array where each element is a single value indexed by an integer key.
- A
RunArray
array where run ends are stored usingi16
data type. - Example: Using
collect
- Buffer builder for signed 32-bit integer type.
- A signed 32-bit integer array builder.
- A dictionary array where each element is a single value indexed by an integer key.
- A
RunArray
array where run ends are stored usingi32
data type. - Example: Using
collect
- Buffer builder for signed 64-bit integer type.
- A signed 64-bit integer array builder.
- A dictionary array where each element is a single value indexed by an integer key.
- A
RunArray
array where run ends are stored usingi64
data type. - An array where each element is a “calendar” interval days and milliseconds.
- Buffer builder for “calendar” interval in days and milliseconds.
- A “calendar” interval in days and milliseconds array builder.
- An array where each element is a “calendar” interval in months, days, and nanoseconds.
- Buffer builder “calendar” interval in months, days, and nanoseconds.
- A “calendar” interval in months, days, and nanoseconds array builder.
- An array where each element is a “calendar” interval in months.
- Buffer builder for “calendar” interval in months.
- A “calendar” interval in months array builder.
- An array where each element contains 0 or more bytes. The byte length of each element is represented by an i64.
- A binary array builder with i64 offsets
- Array builder for
DictionaryArray
that stores large binary. For example to map a set of byte indices to binary values. Note that the use of aHashMap
here will not scale to very large arrays or result in an ordered dictionary. - Array builder for
RunArray
that encodes large binary values(LargeBinaryType
). See documentation ofBinaryRunBuilder
for an example. - A list array where each element is a variable-sized sequence of values with the same type whose memory offsets between elements are represented by a i64.
- Builder for
LargeListArray
s (i64 offsets) - An array where each element is a variable-sized sequence of bytes representing a string whose maximum length (in bytes) is represented by a i64.
- A string array builder with i64 offsets
- Array builder for
DictionaryArray
that stores large Strings. For example to map a set of byte indices to String values. Note that the use of aHashMap
here will not scale to very large arrays or result in an ordered dictionary. - Array builder for
RunArray
that encodes large strings (LargeUtf8Type
). SeeStringRunBuilder
for an example. - A list array where each element is a variable-sized sequence of values with the same type whose memory offsets between elements are represented by a i32.
- Builder for
ListArray
s (i32 offsets) - an iterator that returns Some(T) or None, that can be used on any PrimitiveArray
- An array where each element is a variable-sized sequence of bytes representing a string whose maximum length (in bytes) is represented by a i32.
- A string array builder with i32 offsets
- Array builder for
DictionaryArray
that stores Strings. For example to map a set of byte indices to String values. Note that the use of aHashMap
here will not scale to very large arrays or result in an ordered dictionary. - An array where each element is of 32-bit type representing time elapsed in milliseconds since midnight.
- Buffer builder for 32-bit elaspsed time since midnight of millisecond unit.
- A 32-bit elaspsed time in milliseconds array builder.
- An array where each element is of 32-bit type representing time elapsed in seconds since midnight.
- Buffer builder for 32-bit elaspsed time since midnight of second unit.
- A 32-bit elaspsed time in seconds array builder.
- An array where each element is of 64-bit type representing time elapsed in microseconds since midnight.
- Buffer builder for 64-bit elaspsed time since midnight of microsecond unit.
- A 64-bit elaspsed time in microseconds array builder.
- An array where each element is of 64-bit type representing time elapsed in nanoseconds since midnight.
- Buffer builder for 64-bit elaspsed time since midnight of nanosecond unit.
- A 64-bit elaspsed time in nanoseconds array builder.
- A primitive array where each element is of type
TimestampMicrosecondType.
See examples forTimestampSecondArray.
- Buffer builder for timestamp type of microsecond unit.
- A timestamp microsecond array builder.
- A primitive array where each element is of type
TimestampMillisecondType.
See examples forTimestampSecondArray.
- Buffer builder for timestamp type of millisecond unit.
- A timestamp millisecond array builder.
- A primitive array where each element is of type
TimestampNanosecondType.
See examples forTimestampSecondArray.
- Buffer builder for timestamp type of nanosecond unit.
- A timestamp nanosecond array builder.
- A primitive array where each element is of type TimestampSecondType. See also
Timestamp
. - Buffer builder for timestamp type of second unit.
- A timestamp second array builder.
- Example: Using
collect
- Buffer builder for usigned 8-bit integer type.
- An usigned 8-bit integer array builder.
- A dictionary array where each element is a single value indexed by an integer key.
- Example: Using
collect
- Buffer builder for usigned 16-bit integer type.
- An usigned 16-bit integer array builder.
- A dictionary array where each element is a single value indexed by an integer key.
- Example: Using
collect
- Buffer builder for usigned 32-bit integer type.
- An usigned 32-bit integer array builder.
- A dictionary array where each element is a single value indexed by an integer key.
- Example: Using
collect
- Buffer builder for usigned 64-bit integer type.
- An usigned 64-bit integer array builder.
- A dictionary array where each element is a single value indexed by an integer key.