Expand description
Low-level buffer abstractions for Apache Arrow Rust
§Byte Storage abstractions
MutableBuffer: Raw memory buffer that can be mutated and grownBuffer: Immutable buffer that is shared across threads
§Typed Abstractions
There are also several wrappers over Buffer with methods for
easier manipulation:
BooleanBuffer: Bitmasks (buffer of packed bits)NullBuffer: Arrow null (validity) bitmaps (BooleanBufferwith extra utilities)ScalarBuffer<T>: Typed buffer for primitive types (e.g.,i32,f64)OffsetBuffer<O>: Offsets used in variable-length types (e.g., strings, lists)RunEndBuffer<E>: Run-ends used in run-encoded data
§Platform Support
Only little-endian platforms are officially supported and tested in CI. Big-endian platforms are not tested in CI and may not work correctly. Fixes for big-endian platforms are welcome and handled on a best-effort basis, but compatibility is not guaranteed.
Re-exports§
Modules§
- alloc
- Defines the low-level
AllocationAPI for shared memory regions - bit_
chunk_ iterator - Types for iterating over bitmasks in 64-bit chunks
- bit_
iterator - Types for iterating over packed bitmasks
- bit_
mask - Utils for working with packed bit masks
- bit_
util - Utils for working with bits
- buffer
- Types of shared memory region
- builder
- Buffer builders
Structs§
- AddIndicator
Divisor I32 - A
i32divisor whose magnitude isn’t a power of two, seeDivisorI32 - AddIndicator
Divisor I64 - A
i64divisor whose magnitude isn’t a power of two, seeDivisorI64 - AddIndicator
Divisor U32 - A
u32divisor that isn’t a power of two, seeDivisorU32 - AddIndicator
Divisor U64 - A
u64divisor that isn’t a power of two, seeDivisorU64 - Down
Divisor U32 - A
u32divisor whose rounded-up multiplier isn’t exact in N bits, seeDivisorU32 - Interval
DayTime - Value of an IntervalDayTime array
- Interval
Month DayNano - Value of an IntervalMonthDayNano array
- Overflow
Error - An arithmetic overflow.
- Power
OfTwo Divisor I32 - A
i32divisor whose magnitude is a power of two, seeDivisorI32 - Power
OfTwo Divisor I64 - A
i64divisor whose magnitude is a power of two, seeDivisorI64 - Power
OfTwo Divisor U32 - A
u32divisor that is a power of two, seeDivisorU32 - Power
OfTwo Divisor U64 - A
u64divisor that is a power of two, seeDivisorU64 - Slice
OutOf Bounds Error - A slice that does not fit in the buffer it is taken from.
- Tracking
Memory Pool pool - A simple
MemoryPoolthat reports the total memory usage - UpDivisor
U32 - A
u32divisor whose rounded-up multiplier is exact in N bits, seeDivisorU32 - UpDivisor
U64 - A
u64divisor whose rounded-up multiplier is exact in N bits, seeDivisorU64 - i256
- A signed 256-bit integer
Enums§
- Divisor
I32 - A
i32divisor, precomputed once to divide many values by it - Divisor
I64 - A
i64divisor, precomputed once to divide many values by it - Divisor
U32 - A
u32divisor, precomputed once to divide many values by it - Divisor
U64 - A
u64divisor, precomputed once to divide many values by it
Traits§
- Arrow
Native Type - Trait expressing a Rust type that has the same in-memory representation as Arrow.
- Memory
Pool pool - A pool of memory that can be reserved and released.
- Memory
Reservation pool - A memory reservation within a
MemoryPoolthat is freed on drop - ToByte
Slice - Allows conversion from supported Arrow types to a byte slice.