Expand description
§Minarrow - The Latest and Greatest Rust Innovation Since One-Nanosecond Ago
Next-generation Rust library implementing the Apache Arrow zero-copy memory layout — built for high-performance computing, streaming, embedded systems, and for Rust developers who demand simple, powerful, and efficient data.
§Standout features:
- Very fast compile times (< 1.5 s standard features, < 0.15 s rebuilds)
- Automatic 64-byte SIMD alignment
- Exceptional runtime performance - see benchmarks below
- Cohesive, well-documented API with extensive coverage
- FFI compatibility, including easy to_apache_arrow() and to_polars()
§Coming soon
- Lightstream-IO — IPC streaming and Tokio async integration
- Kernels — Large library of SIMD-ready computation kernels
Both integrate seamlessly with Minarrow.
Interested in contributing? Reach out.
§Thank you
Thank you for using Minarrow. If you find it useful, share it with a friend or colleague.
§Copyright
Copyright © 2025 Peter Garfield Bower. All rights reserved. Licensed under the MIT license.
§Arrow compatibility
This library closely follows the Apache Arrow format,
but simplifies certain APIs while maintaining strong compatibility.
Minarrow uses the documented memory layouts for all array types, but
consolidates logical types in some cases when they share identical physical storage
(e.g., DateTimeArray). Additional types are provided where they add value.
Learn more about Apache Arrow at: https://arrow.apache.org/overview/
Minarrow is not affiliated with Apache Arrow or the Apache Software Foundation.
Apache Arrow is a registered trademark of the ASF, referenced here under
fair use.
§Editions
Requires Rust nightly for modern, yet unstable, features such as allocator_api.
§Benchmarks
Sum of 1 billion sequential integers starting at 0.
Intel(R) Core(TM) Ultra 7 155H | x86_64 | 22 CPUs
Averaged over 1,000 runs (release).
§No SIMD
(n=1000, lanes=4, iters=1000)
| Case | Avg time |
|---|---|
| Integer (i64) | |
raw vec: Vec<i64> | 85 ns |
minarrow direct: IntegerArray | 88 ns |
arrow-rs struct: Int64Array | 147 ns |
minarrow enum: IntegerArray | 124 ns |
arrow-rs dyn: Int64Array | 181 ns |
| Float (f64) | |
raw vec: Vec<f64> | 475 ns |
minarrow direct: FloatArray | 476 ns |
arrow-rs struct: Float64Array | 527 ns |
minarrow enum: FloatArray | 507 ns |
arrow-rs dyn: Float64Array | 1.952 µs |
§SIMD
(n=1000, lanes=4, iters=1000)
| Case | Avg (ns) |
|---|---|
raw vec: Vec<i64> | 64 |
raw vec64: Vec64<i64> | 55 |
minarrow direct: IntegerArray | 88 |
arrow-rs struct: Int64Array | 162 |
minarrow enum: IntegerArray | 170 |
arrow-rs dyn: Int64Array | 173 |
raw vec: Vec<f64> | 57 |
raw vec64: Vec64<f64> | 58 |
minarrow direct: FloatArray | 91 |
arrow-rs struct: Float64Array | 181 |
minarrow enum: FloatArray | 180 |
arrow-rs dyn: Float64Array | 196 |
§SIMD + Rayon
(n=1000, lanes=4, iters=1000)
| Case | Time (ms) |
|---|---|
SIMD + Rayon IntegerArray<i64> | 113.874 |
SIMD + Rayon FloatArray<f64> | 114.095 |
§Other benchmark factors
Vec
Vec64
The construction delta is not included in the benchmark timings above.
Re-exports§
pub use aliases::BytesLength;pub use aliases::DictLength;pub use aliases::Length;pub use aliases::Offset;pub use aliases::ArrayVT;pub use aliases::BitmaskVT;pub use aliases::StringAVT;pub use aliases::StringAVTExt;pub use aliases::CategoricalAVT;pub use aliases::CategoricalAVTExt;pub use aliases::IntegerAVT;pub use aliases::FloatAVT;pub use aliases::BooleanAVT;pub use aliases::DatetimeAVT;pub use enums::time_units::IntervalUnit;pub use enums::time_units::TimeUnit;pub use enums::value::Value;pub use enums::scalar::Scalar;pub use enums::array::Array;pub use enums::collections::numeric_array::NumericArray;pub use enums::collections::temporal_array::TemporalArray;pub use enums::collections::text_array::TextArray;pub use structs::buffer::Buffer;pub use structs::bitmask::Bitmask;pub use structs::views::bitmask_view::BitmaskV;pub use structs::chunked::super_array::SuperArray;pub use structs::chunked::super_table::SuperTable;pub use structs::views::chunked::super_array_view::SuperArrayV;pub use structs::views::chunked::super_table_view::SuperTableV;pub use structs::views::array_view::ArrayV;pub use structs::views::collections::numeric_array_view::NumericArrayV;pub use structs::views::collections::temporal_array_view::TemporalArrayV;pub use structs::views::collections::text_array_view::TextArrayV;pub use structs::field::Field;pub use structs::field_array::FieldArray;pub use structs::table::Table;pub use structs::cube::Cube;pub use structs::matrix::Matrix;pub use structs::variants::boolean::BooleanArray;pub use structs::variants::categorical::CategoricalArray;pub use structs::variants::datetime::DatetimeArray;pub use structs::variants::float::FloatArray;pub use structs::variants::integer::IntegerArray;pub use structs::variants::string::StringArray;pub use structs::vec64::Vec64;pub use structs::views::table_view::TableV;pub use traits::masked_array::MaskedArray;pub use traits::print::Print;pub use traits::type_unions::Float;pub use traits::type_unions::Integer;pub use traits::type_unions::Numeric;pub use traits::type_unions::Primitive;pub use ffi::arrow_dtype::ArrowType;
Modules§
- aliases
- Aliases & View Tuples - Lightweight Tuple Views and Fast-To-Type Aliases
- conversions
- Conversions & Views - Most To/From Boilerplate Implements Here
- enums
- ffi
- macros
- Macros — Streamlines repetitive Array implementations to avoid duplication
- structs
- traits
- utils
- Utilities - Internal Helper Utilities
Macros§
- arr_
bool - arr_
bool_ opt - arr_
cat8 - arr_
cat8_ opt - arr_
cat16 - arr_
cat32 - arr_
cat64 - arr_
cat16_ opt - arr_
cat32_ opt - arr_
cat64_ opt - arr_f32
- arr_f64
- arr_
f32_ opt - arr_
f64_ opt - arr_i8
- arr_
i8_ opt - arr_i16
- arr_i32
- arr_i64
- arr_
i16_ opt - arr_
i32_ opt - arr_
i64_ opt - arr_
str32 - arr_
str64 - arr_
str32_ opt - arr_
str64_ opt - arr_u8
- arr_
u8_ opt - arr_u16
- arr_u32
- arr_u64
- arr_
u16_ opt - arr_
u32_ opt - arr_
u64_ opt - has_
nulls - impl_
arc_ masked_ array - Overview
- impl_
array_ ref_ deref - Implements AsRef, AsMut, Deref, and DerefMut for standard array types with
.data: Vec64<T>. This macro is for value buffers only, not dictionary arrays or string offset views. - impl_
from_ vec_ primitive - Implement
from_vec+from_std_vecfor the “numeric-shaped” arrays (IntegerArray / FloatArray / DatetimeArray). - impl_
masked_ array - Implements the MaskedArray
trait for the given struct and bound. - impl_
numeric_ array_ constructors - Implements standard constructors for columnar array types with data, null_mask, and PhantomData fields.
- impl_
usize_ conversions - Implements usize conversions for integers
- match_
array - Reduces matching boilerplate when all positive paths share the outcome
- vec64