#[repr(C, align(64))]pub enum NumericArray {
Int8(Arc<IntegerArray<i8>>),
Int16(Arc<IntegerArray<i16>>),
Int32(Arc<IntegerArray<i32>>),
Int64(Arc<IntegerArray<i64>>),
UInt8(Arc<IntegerArray<u8>>),
UInt16(Arc<IntegerArray<u16>>),
UInt32(Arc<IntegerArray<u32>>),
UInt64(Arc<IntegerArray<u64>>),
Float32(Arc<FloatArray<f32>>),
Float64(Arc<FloatArray<f64>>),
Null,
}Expand description
§NumericArray
Unified numerical array container
§Purpose
Exists to unify numerical operations, simplify API’s and streamline user ergonomics.
§Usage:
- It is accessible from
Arrayusing.num(), and provides typed variant access via for e.g.,.i64(), so one can drill down to the required granularity viamyarr.num().i64() - This streamlines function implementations,
and, despite the additional
enumlayer, matching lanes in many real-world scenarios. This is because one can for e.g., unify a function signature withimpl Into<NumericArray>, and all of the subtypes, plusArrayandNumericalArray, all qualify. - Additionally, you can then use one
Integerimplementation on the enum dispatch arm for allIntegervariants, or, in many cases, for the entire numeric arm when they are the same.
§Typecasting behaviour
- If the enum already holds the given type (which should be known at compile-time),
then using accessors like
.i32()is zero-cost, as it transfers ownership. - If you want to keep the original, of course use
.clone()beforehand. - If you use an accessor to a different base type, e.g.,
.f32()when it’s a.int32()already in the enum, it will convert it. Therefore, be mindful of performance when this occurs.
§Also see:
- Under crate::traits::type_unions , we additionally
include minimal
Integer,Float,NumericandPrimitivetraits that for which the base Rust primitive types already qualify. These are loose wrappers over thenum-traitscrate to help improve type ergonomics when traits are required, but without requiring any downcasting.
Variants§
Int8(Arc<IntegerArray<i8>>)
Int16(Arc<IntegerArray<i16>>)
Int32(Arc<IntegerArray<i32>>)
Int64(Arc<IntegerArray<i64>>)
UInt8(Arc<IntegerArray<u8>>)
UInt16(Arc<IntegerArray<u16>>)
UInt32(Arc<IntegerArray<u32>>)
UInt64(Arc<IntegerArray<u64>>)
Float32(Arc<FloatArray<f32>>)
Float64(Arc<FloatArray<f64>>)
Null
Implementations§
Source§impl NumericArray
impl NumericArray
Sourcepub fn append_array(&mut self, other: &Self)
pub fn append_array(&mut self, other: &Self)
Appends all values (and null mask if present) from other into self.
Panics if the two arrays are of different variants or incompatible types.
This function uses copy-on-write semantics for arrays wrapped in Arc.
If self is the only owner of its data, appends are performed in place without copying.
If the array data is shared (Arc reference count > 1), the data is first cloned
(so the mutation does not affect other owners), and the append is then performed on the unique copy.
This ensures that calling append_array never mutates data referenced elsewhere,
but also avoids unnecessary cloning when the data is uniquely owned.
Sourcepub fn i32(self) -> Result<IntegerArray<i32>, MinarrowError>
pub fn i32(self) -> Result<IntegerArray<i32>, MinarrowError>
Convert to IntegerArray
Sourcepub fn i64(self) -> Result<IntegerArray<i64>, MinarrowError>
pub fn i64(self) -> Result<IntegerArray<i64>, MinarrowError>
Convert to IntegerArray
Examples found in repository?
examples/benchmarks/hotloop_benchmark_std.rs (line 102)
33 pub(crate) fn run_benchmark() {
34 // ----------- Raw Vec<i64> -----------
35 let raw_vec: Vec<i64> = (0..N as i64).collect();
36 let start = Instant::now();
37 let mut acc = 0i64;
38 for &v in &raw_vec {
39 acc += v;
40 }
41 let dur_vec_i64 = start.elapsed();
42 println!("raw vec: Vec<i64> sum = {}, {:?}", acc, dur_vec_i64);
43 black_box(acc);
44 std::mem::drop(raw_vec);
45
46 // ----------- Raw Vec64<i64> -----------
47 let raw_vec: Vec64<i64> = (0..N as i64).collect();
48 let start = Instant::now();
49 let mut acc = 0i64;
50 for &v in &raw_vec {
51 acc += v;
52 }
53 let dur_vec_i64 = start.elapsed();
54 println!("raw vec: Vec64<i64> sum = {}, {:?}", acc, dur_vec_i64);
55 black_box(acc);
56 std::mem::drop(raw_vec);
57
58 // ----------- Minarrow i64 (direct struct, no enum) -----------
59 let min_data: Vec64<i64> = (0..N as i64).collect();
60 let start = Instant::now();
61 let int_arr = IntegerArray {
62 data: Buffer::from(min_data),
63 null_mask: None,
64 };
65 let mut acc = 0i64;
66 let slice = int_arr.data.as_slice();
67 for &v in slice {
68 acc += v;
69 }
70 let dur_minarrow_direct_i64 = start.elapsed();
71 println!(
72 "minarrow direct: IntegerArray sum = {}, {:?}",
73 acc, dur_minarrow_direct_i64
74 );
75 black_box(acc);
76 std::mem::drop(int_arr);
77
78 // ----------- Arrow i64 (struct direct) -----------
79 let data: Vec<i64> = (0..N as i64).collect();
80 let start = Instant::now();
81 let arr = ArrowI64Array::from(data);
82 let mut acc = 0i64;
83 for i in 0..arr.len() {
84 acc += arr.value(i);
85 }
86 let dur_arrow_struct_i64 = start.elapsed();
87 println!(
88 "arrow-rs struct: Int64Array sum = {}, {:?}",
89 acc, dur_arrow_struct_i64
90 );
91 black_box(acc);
92 std::mem::drop(arr);
93
94 // ----------- Minarrow i64 (enum) -----------
95 let min_data: Vec64<i64> = (0..N as i64).collect();
96 let start = Instant::now();
97 let array = Array::NumericArray(NumericArray::Int64(Arc::new(IntegerArray {
98 data: Buffer::from(min_data),
99 null_mask: None,
100 })));
101 let mut acc = 0i64;
102 let int_arr = array.num().i64().unwrap();
103 let slice = int_arr.data.as_slice();
104 for &v in slice {
105 acc += v;
106 }
107 let dur_minarrow_enum_i64 = start.elapsed();
108 println!(
109 "minarrow enum: IntegerArray sum = {}, {:?}",
110 acc, dur_minarrow_enum_i64
111 );
112 black_box(acc);
113 std::mem::drop(int_arr);
114
115 // ----------- Arrow i64 (dynamic) -----------
116 let data_dyn: Vec<i64> = (0..N as i64).collect();
117 let start = Instant::now();
118 let arr_dyn: ArrayRef = Arc::new(ArrowI64Array::from(data_dyn));
119 let mut acc = 0i64;
120 if let Some(int) = arr_dyn.as_any().downcast_ref::<ArrowI64Array>() {
121 for i in 0..int.len() {
122 acc += int.value(i);
123 }
124 }
125 let dur_arrow_dyn_i64 = start.elapsed();
126 println!(
127 "arrow-rs dyn: ArrayRef Int64Array sum = {}, {:?}",
128 acc, dur_arrow_dyn_i64
129 );
130 black_box(acc);
131 std::mem::drop(arr_dyn);
132
133 // ----------- Raw Vec<f64> -----------
134 let raw_vec: Vec<f64> = (0..N as i64).map(|x| x as f64).collect();
135 let start = Instant::now();
136 let mut acc = 0.0f64;
137 for &v in &raw_vec {
138 acc += v;
139 }
140 let dur_vec_f64 = start.elapsed();
141 println!("raw vec: Vec<f64> sum = {}, {:?}", acc, dur_vec_f64);
142 black_box(acc);
143 std::mem::drop(raw_vec);
144
145 // ----------- Raw Vec64<f64> -----------
146 let raw_vec: Vec64<f64> = (0..N as i64).map(|x| x as f64).collect();
147 let start = Instant::now();
148 let mut acc = 0.0f64;
149 for &v in &raw_vec {
150 acc += v;
151 }
152 let dur_vec_f64 = start.elapsed();
153 println!("raw vec: Vec<f64> sum = {}, {:?}", acc, dur_vec_f64);
154 black_box(acc);
155 std::mem::drop(raw_vec);
156
157 // ----------- Minarrow f64 (direct struct, no enum) -----------
158 let min_data_f64: Vec64<f64> = (0..N as i64).map(|x| x as f64).collect();
159 let start = Instant::now();
160 let float_arr = FloatArray {
161 data: Buffer::from(min_data_f64),
162 null_mask: None,
163 };
164 let mut acc = 0.0f64;
165 let slice = float_arr.data.as_slice();
166 for &v in slice {
167 acc += v;
168 }
169 let dur_minarrow_direct_f64 = start.elapsed();
170 println!(
171 "minarrow direct: FloatArray sum = {}, {:?}",
172 acc, dur_minarrow_direct_f64
173 );
174 black_box(acc);
175 std::mem::drop(float_arr);
176
177 // ----------- Arrow f64 (struct direct) -----------
178 let data_f64: Vec<f64> = (0..N as i64).map(|x| x as f64).collect();
179 let start = Instant::now();
180 let arr = ArrowF64Array::from(data_f64);
181 let mut acc = 0.0f64;
182 for i in 0..arr.len() {
183 acc += arr.value(i);
184 }
185 let dur_arrow_struct_f64 = start.elapsed();
186 println!(
187 "arrow-rs struct: Float64Array sum = {}, {:?}",
188 acc, dur_arrow_struct_f64
189 );
190 black_box(acc);
191 std::mem::drop(arr);
192
193 // ----------- Minarrow f64 (enum) -----------
194 let min_data_f64: Vec64<f64> = (0..N as i64).map(|x| x as f64).collect();
195 let start = Instant::now();
196 let array = Array::NumericArray(NumericArray::Float64(Arc::new(FloatArray {
197 data: Buffer::from(min_data_f64),
198 null_mask: None,
199 })));
200 let mut acc = 0.0f64;
201 let float_arr = array.num().f64().unwrap();
202 let slice = float_arr.data.as_slice();
203 for &v in slice {
204 acc += v;
205 }
206 let dur_minarrow_enum_f64 = start.elapsed();
207 println!(
208 "minarrow enum: FloatArray sum = {}, {:?}",
209 acc, dur_minarrow_enum_f64
210 );
211 black_box(acc);
212 std::mem::drop(float_arr);
213
214 // ----------- Arrow f64 (dynamic) -----------
215 let data_f64: Vec<f64> = (0..N as i64).map(|x| x as f64).collect();
216 let start = Instant::now();
217 let arr: ArrayRef = Arc::new(ArrowF64Array::from(data_f64));
218 let mut acc = 0.0f64;
219 if let Some(f) = arr.as_any().downcast_ref::<ArrowF64Array>() {
220 for i in 0..f.len() {
221 acc += f.value(i);
222 }
223 }
224 let dur_arrow_dyn_f64 = start.elapsed();
225 println!(
226 "arrow-rs dyn: Float64Array sum = {}, {:?}",
227 acc, dur_arrow_dyn_f64
228 );
229 black_box(acc);
230 std::mem::drop(arr);
231 }More examples
examples/benchmarks/hotloop_benchmark_avg_std.rs (line 83)
36 pub(crate) fn run_benchmark() {
37 let mut total_arrow_dyn_i64 = std::time::Duration::ZERO;
38 let mut total_arrow_struct_i64 = std::time::Duration::ZERO;
39 let mut total_minarrow_enum_i64 = std::time::Duration::ZERO;
40 let mut total_minarrow_direct_i64 = std::time::Duration::ZERO;
41 let mut total_vec_i64 = std::time::Duration::ZERO;
42 let mut total_arrow_dyn_f64 = std::time::Duration::ZERO;
43 let mut total_arrow_struct_f64 = std::time::Duration::ZERO;
44 let mut total_minarrow_enum_f64 = std::time::Duration::ZERO;
45 let mut total_minarrow_direct_f64 = std::time::Duration::ZERO;
46 let mut total_vec_f64 = std::time::Duration::ZERO;
47
48 for _ in 0..ITERATIONS {
49 // ----------- Arrow i64 (dynamic) -----------
50 let data: Vec<i64> = (0..N as i64).collect();
51 let start = Instant::now();
52 let arr: ArrayRef = Arc::new(ArrowI64Array::from(data));
53 let mut acc = 0i64;
54 if let Some(int) = arr.as_any().downcast_ref::<ArrowI64Array>() {
55 for i in 0..int.len() {
56 acc += int.value(i);
57 }
58 }
59 let dur_arrow_dyn_i64 = start.elapsed();
60 total_arrow_dyn_i64 += dur_arrow_dyn_i64;
61 black_box(acc);
62
63 // ----------- Arrow i64 (struct direct) -----------
64 let data: Vec<i64> = (0..N as i64).collect();
65 let start = Instant::now();
66 let arr = ArrowI64Array::from(data);
67 let mut acc = 0i64;
68 for i in 0..arr.len() {
69 acc += arr.value(i);
70 }
71 let dur_arrow_struct_i64 = start.elapsed();
72 total_arrow_struct_i64 += dur_arrow_struct_i64;
73 black_box(acc);
74
75 // ----------- Minarrow i64 (enum) -----------
76 let min_data: Vec64<i64> = (0..N as i64).collect();
77 let start = Instant::now();
78 let array = Array::NumericArray(NumericArray::Int64(Arc::new(IntegerArray {
79 data: Buffer::from(min_data),
80 null_mask: None,
81 })));
82 let mut acc = 0i64;
83 let int_arr = array.num().i64().unwrap();
84 let slice = int_arr.data.as_slice();
85 for &v in slice {
86 acc += v;
87 }
88 let dur_minarrow_enum_i64 = start.elapsed();
89 total_minarrow_enum_i64 += dur_minarrow_enum_i64;
90 black_box(acc);
91
92 // ----------- Minarrow i64 (direct struct, no enum) -----------
93 let min_data: Vec64<i64> = (0..N as i64).collect();
94 let start = Instant::now();
95 let int_arr = IntegerArray {
96 data: Buffer::from(min_data),
97 null_mask: None,
98 };
99 let mut acc = 0i64;
100 let slice = int_arr.data.as_slice();
101 for &v in slice {
102 acc += v;
103 }
104 let dur_minarrow_direct_i64 = start.elapsed();
105 total_minarrow_direct_i64 += dur_minarrow_direct_i64;
106 black_box(acc);
107
108 // ----------- Raw Vec<i64> -----------
109 let raw_vec: Vec<i64> = (0..N as i64).collect();
110 let start = Instant::now();
111 let mut acc = 0i64;
112 for &v in &raw_vec {
113 acc += v;
114 }
115 let dur_vec_i64 = start.elapsed();
116 total_vec_i64 += dur_vec_i64;
117 black_box(acc);
118
119 // ----------- Arrow f64 (dynamic) -----------
120 let data_f64: Vec<f64> = (0..N as i64).map(|x| x as f64).collect();
121 let start = Instant::now();
122 let arr: ArrayRef = Arc::new(ArrowF64Array::from(data_f64));
123 let mut acc = 0.0f64;
124 if let Some(f) = arr.as_any().downcast_ref::<ArrowF64Array>() {
125 for i in 0..f.len() {
126 acc += f.value(i);
127 }
128 }
129 let dur_arrow_dyn_f64 = start.elapsed();
130 total_arrow_dyn_f64 += dur_arrow_dyn_f64;
131 black_box(acc);
132
133 // ----------- Arrow f64 (struct direct) -----------
134 let data_f64: Vec<f64> = (0..N as i64).map(|x| x as f64).collect();
135 let start = Instant::now();
136 let arr = ArrowF64Array::from(data_f64);
137 let mut acc = 0.0f64;
138 for i in 0..arr.len() {
139 acc += arr.value(i);
140 }
141 let dur_arrow_struct_f64 = start.elapsed();
142 total_arrow_struct_f64 += dur_arrow_struct_f64;
143 black_box(acc);
144
145 // ----------- Minarrow f64 (enum) -----------
146 let min_data_f64: Vec64<f64> = (0..N as i64).map(|x| x as f64).collect();
147 let start = Instant::now();
148 let array = Array::NumericArray(NumericArray::Float64(Arc::new(FloatArray {
149 data: Buffer::from(min_data_f64),
150 null_mask: None,
151 })));
152 let mut acc = 0.0f64;
153 let float_arr = array.num().f64().unwrap();
154 let slice = float_arr.data.as_slice();
155 for &v in slice {
156 acc += v;
157 }
158 let dur_minarrow_enum_f64 = start.elapsed();
159 total_minarrow_enum_f64 += dur_minarrow_enum_f64;
160 black_box(acc);
161
162 // ----------- Minarrow f64 (direct struct, no enum) -----------
163 let min_data_f64: Vec64<f64> = (0..N as i64).map(|x| x as f64).collect();
164 let start = Instant::now();
165 let float_arr = FloatArray {
166 data: Buffer::from(min_data_f64),
167 null_mask: None,
168 };
169 let mut acc = 0.0f64;
170 let slice = float_arr.data.as_slice();
171 for &v in slice {
172 acc += v;
173 }
174 let dur_minarrow_direct_f64 = start.elapsed();
175 total_minarrow_direct_f64 += dur_minarrow_direct_f64;
176 black_box(acc);
177
178 // ----------- Raw Vec<f64> -----------
179 let raw_vec: Vec<f64> = (0..N as i64).map(|x| x as f64).collect();
180 let start = Instant::now();
181 let mut acc = 0.0f64;
182 for &v in &raw_vec {
183 acc += v;
184 }
185 let dur_vec_f64 = start.elapsed();
186 total_vec_f64 += dur_vec_f64;
187 black_box(acc);
188 }
189
190 println!("Averaged Results from {} runs:", ITERATIONS);
191 println!("---------------------------------");
192
193 let avg_vec_i64 = total_vec_i64.as_nanos() as f64 / ITERATIONS as f64;
194 let avg_minarrow_direct_i64 =
195 total_minarrow_direct_i64.as_nanos() as f64 / ITERATIONS as f64;
196 let avg_arrow_struct_i64 = total_arrow_struct_i64.as_nanos() as f64 / ITERATIONS as f64;
197 let avg_minarrow_enum_i64 = total_minarrow_enum_i64.as_nanos() as f64 / ITERATIONS as f64;
198 let avg_arrow_dyn_i64 = total_arrow_dyn_i64.as_nanos() as f64 / ITERATIONS as f64;
199
200 let avg_vec_f64 = total_vec_f64.as_nanos() as f64 / ITERATIONS as f64;
201 let avg_minarrow_direct_f64 =
202 total_minarrow_direct_f64.as_nanos() as f64 / ITERATIONS as f64;
203 let avg_arrow_struct_f64 = total_arrow_struct_f64.as_nanos() as f64 / ITERATIONS as f64;
204 let avg_minarrow_enum_f64 = total_minarrow_enum_f64.as_nanos() as f64 / ITERATIONS as f64;
205 let avg_arrow_dyn_f64 = total_arrow_dyn_f64.as_nanos() as f64 / ITERATIONS as f64;
206
207 println!(
208 "raw vec: Vec<i64> avg = {} (n={})",
209 fmt_duration_ns(avg_vec_i64),
210 ITERATIONS
211 );
212 println!(
213 "minarrow direct: IntegerArray avg = {} (n={})",
214 fmt_duration_ns(avg_minarrow_direct_i64),
215 ITERATIONS
216 );
217 println!(
218 "arrow-rs struct: Int64Array avg = {} (n={})",
219 fmt_duration_ns(avg_arrow_struct_i64),
220 ITERATIONS
221 );
222 println!();
223 println!(
224 "minarrow enum: IntegerArray avg = {} (n={})",
225 fmt_duration_ns(avg_minarrow_enum_i64),
226 ITERATIONS
227 );
228 println!(
229 "arrow-rs dyn: Int64Array avg = {} (n={})",
230 fmt_duration_ns(avg_arrow_dyn_i64),
231 ITERATIONS
232 );
233 println!();
234 println!(
235 "raw vec: Vec<f64> avg = {} (n={})",
236 fmt_duration_ns(avg_vec_f64),
237 ITERATIONS
238 );
239 println!(
240 "minarrow direct: FloatArray avg = {} (n={})",
241 fmt_duration_ns(avg_minarrow_direct_f64),
242 ITERATIONS
243 );
244 println!(
245 "arrow-rs struct: Float64Array avg = {} (n={})",
246 fmt_duration_ns(avg_arrow_struct_f64),
247 ITERATIONS
248 );
249 println!();
250 println!(
251 "minarrow enum: FloatArray avg = {} (n={})",
252 fmt_duration_ns(avg_minarrow_enum_f64),
253 ITERATIONS
254 );
255 println!(
256 "arrow-rs dyn: Float64Array avg = {} (n={})",
257 fmt_duration_ns(avg_arrow_dyn_f64),
258 ITERATIONS
259 );
260 }examples/benchmarks/hotloop_benchmark_simd.rs (line 266)
188 pub fn run_benchmark(n: usize, simd_lanes: usize) {
189 // ----------- Integer (i64) tests -----------
190
191 let data: Vec<i64> = (0..n as i64).collect();
192 black_box(simd_sum_i64_runtime(&data[..], simd_lanes)); // warmup, ignore result
193
194 println!("|------------ Integer Tests ------------ |\n");
195 // Raw Vec<i64>
196 // Sometimes this will randomly align, other times it will not.
197 let data: Vec<i64> = (0..n as i64).collect();
198 let start = Instant::now();
199 let slice = &data[..];
200 let sum = simd_sum_i64_runtime(slice, simd_lanes);
201 let dur_vec = start.elapsed();
202 println!("raw vec: Vec<i64> sum = {}, {:?}", sum, dur_vec);
203 let v_aligned =
204 (&data[0] as *const i64 as usize) % std::mem::align_of::<Simd<i64, SIMD_LANES>>() == 0;
205 black_box(sum);
206
207 // Raw Vec64<i64>
208 let data: Vec64<i64> = (0..n as i64).collect();
209 let start = Instant::now();
210 let slice = &data[..];
211 let sum = simd_sum_i64_runtime(slice, simd_lanes);
212 let dur_vec = start.elapsed();
213 println!("raw vec64: Vec64<i64> sum = {}, {:?}", sum, dur_vec);
214 let v64_aligned =
215 (&data[0] as *const i64 as usize) % std::mem::align_of::<Simd<i64, SIMD_LANES>>() == 0;
216 black_box(sum);
217
218 // Minarrow i64 (direct struct)
219 let data: Vec64<i64> = (0..n as i64).collect();
220 let data_copy = data.clone();
221
222 let start = Instant::now();
223 let int_arr = IntegerArray {
224 data: Buffer::from(data),
225 null_mask: None,
226 };
227 let slice = &int_arr[..];
228 let sum = simd_sum_i64_runtime(slice, simd_lanes);
229 let dur_minarrow_direct = start.elapsed();
230 println!(
231 "minarrow direct: IntegerArray sum = {}, {:?}",
232 sum, dur_minarrow_direct
233 );
234 let int_array_aligned = (&data_copy[0] as *const i64 as usize)
235 % std::mem::align_of::<Simd<i64, SIMD_LANES>>()
236 == 0;
237 black_box(sum);
238
239 // Arrow i64 (struct direct)
240 let data: Vec<i64> = (0..n as i64).collect();
241 let data_copy = data.clone();
242
243 let start = Instant::now();
244 let arr = ArrowI64Array::from(data);
245 let slice = arr.values();
246 let sum = simd_sum_i64_runtime(slice, simd_lanes);
247 let dur_arrow_struct = start.elapsed();
248 println!(
249 "arrow-rs struct: Int64Array sum = {}, {:?}",
250 sum, dur_arrow_struct
251 );
252 let i64_arrow_aligned = (&data_copy[0] as *const i64 as usize)
253 % std::mem::align_of::<Simd<i64, SIMD_LANES>>()
254 == 0;
255 black_box(sum);
256
257 // Minarrow i64 (enum)
258 let data: Vec64<i64> = (0..n as i64).collect();
259 let data_copy = data.clone();
260
261 let start = Instant::now();
262 let array = Array::NumericArray(NumericArray::Int64(Arc::new(IntegerArray {
263 data: Buffer::from(data),
264 null_mask: None,
265 })));
266 let int_arr = array.num().i64().unwrap();
267 let slice = &int_arr[..];
268 let sum = simd_sum_i64_runtime(slice, simd_lanes);
269 let dur_minarrow_enum = start.elapsed();
270 println!(
271 "minarrow enum: IntegerArray sum = {}, {:?}",
272 sum, dur_minarrow_enum
273 );
274 let arr_int_enum_aligned = (&data_copy[0] as *const i64 as usize)
275 % std::mem::align_of::<Simd<i64, SIMD_LANES>>()
276 == 0;
277 black_box(sum);
278
279 // Arrow i64 (dynamic)
280 let data: Vec<i64> = (0..n as i64).collect();
281 let data_copy = data.clone();
282
283 let start = Instant::now();
284 let arr: ArrayRef = Arc::new(ArrowI64Array::from(data));
285 let slice = if let Some(f) = arr.as_any().downcast_ref::<ArrowI64Array>() {
286 f.values()
287 } else {
288 panic!("downcast failed")
289 };
290 let sum = simd_sum_i64_runtime(slice, simd_lanes);
291 let dur_arrow_dyn_i64 = start.elapsed();
292 println!(
293 "arrow-rs dyn: Int64Array sum = {}, {:?}",
294 sum, dur_arrow_dyn_i64
295 );
296 let array_ref_int_aligned = (&data_copy[0] as *const i64 as usize)
297 % std::mem::align_of::<Simd<i64, SIMD_LANES>>()
298 == 0;
299 black_box(sum);
300 println!("\n");
301
302 // ----------- Float (f64) tests ----------------
303
304 println!("|------------ Float Tests ------------ |\n");
305
306 // Raw Vec<f64>
307 // Sometimes this will randomly align, other times it will not.
308 let data: Vec<f64> = (0..n as i64).map(|x| x as f64).collect();
309 let start = Instant::now();
310 let sum = simd_sum_f64_runtime(&data[..], simd_lanes);
311 let dur_vec_f64 = start.elapsed();
312 println!("raw vec: Vec<f64> sum = {}, {:?}", sum, dur_vec_f64);
313 let v_float_aligned =
314 (&data[0] as *const f64 as usize) % std::mem::align_of::<Simd<f64, SIMD_LANES>>() == 0;
315
316 black_box(sum);
317
318 // Raw Vec64<f64>
319 let data: Vec64<f64> = (0..n as i64).map(|x| x as f64).collect();
320 let start = Instant::now();
321 let sum = simd_sum_f64_runtime(&data[..], simd_lanes);
322 let dur_vec_f64 = start.elapsed();
323 println!("raw vec64: Vec64<f64> sum = {}, {:?}", sum, dur_vec_f64);
324 let v64_float_aligned =
325 (&data[0] as *const f64 as usize) % std::mem::align_of::<Simd<f64, SIMD_LANES>>() == 0;
326
327 black_box(sum);
328
329 // Minarrow f64 (direct struct, SIMD)
330 let data: Vec64<f64> = (0..n as i64).map(|x| x as f64).collect();
331 let data_copy = data.clone();
332
333 let start = Instant::now();
334 let float_arr = FloatArray {
335 data: Buffer::from(data),
336 null_mask: None,
337 };
338 let sum = simd_sum_f64_runtime(&float_arr[..], simd_lanes);
339 let dur_minarrow_direct_f64 = start.elapsed();
340 println!(
341 "minarrow direct: FloatArray sum = {}, {:?}",
342 sum, dur_minarrow_direct_f64
343 );
344 let float_arr_aligned = (&data_copy[0] as *const f64 as usize)
345 % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
346 == 0;
347 black_box(sum);
348
349 // Arrow f64 (struct direct)
350 let data: Vec<f64> = (0..n as i64).map(|x| x as f64).collect();
351 let data_copy = data.clone();
352
353 let start = Instant::now();
354 let arr = ArrowF64Array::from(data);
355 let sum = simd_sum_f64_runtime(arr.values(), simd_lanes);
356 let dur_arrow_struct_f64 = start.elapsed();
357 println!(
358 "arrow-rs struct: Float64Array sum = {}, {:?}",
359 sum, dur_arrow_struct_f64
360 );
361 let arrow_f64_aligned = (&data_copy[0] as *const f64 as usize)
362 % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
363 == 0;
364 black_box(sum);
365
366 // Minarrow f64 (enum)
367 let data: Vec64<f64> = (0..n as i64).map(|x| x as f64).collect();
368 let data_copy = data.clone();
369
370 let start = Instant::now();
371 let array = Array::NumericArray(NumericArray::Float64(Arc::new(FloatArray {
372 data: Buffer::from(data),
373 null_mask: None,
374 })));
375 let float_arr = array.num().f64().unwrap();
376 let sum = simd_sum_f64_runtime(&float_arr[..], simd_lanes);
377 let dur_minarrow_enum_f64 = start.elapsed();
378 println!(
379 "minarrow enum: FloatArray sum = {}, {:?}",
380 sum, dur_minarrow_enum_f64
381 );
382 let float_enum_aligned = (&data_copy[0] as *const f64 as usize)
383 % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
384 == 0;
385 black_box(sum);
386
387 // Arrow f64 (dynamic)
388 let data: Vec<f64> = (0..n as i64).map(|x| x as f64).collect();
389 let data_copy = data.clone();
390
391 let start = Instant::now();
392 let arr: ArrayRef = Arc::new(ArrowF64Array::from(data));
393 let slice = if let Some(f) = arr.as_any().downcast_ref::<ArrowF64Array>() {
394 f.values()
395 } else {
396 panic!("downcast failed")
397 };
398 let sum = simd_sum_f64_runtime(slice, simd_lanes);
399 let dur_arrow_dyn_f64 = start.elapsed();
400 println!(
401 "arrow-rs dyn: Float64Array sum = {}, {:?}",
402 sum, dur_arrow_dyn_f64
403 );
404 let arrow_f64_arr_aligned = (&data_copy[0] as *const f64 as usize)
405 % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
406 == 0;
407 black_box(sum);
408 println!("\n");
409 println!("Verify SIMD pointer alignment for Integer calculations (based on lane width):");
410 println!("Vec<i64> is aligned: {}", v_aligned);
411 println!("Minarrow Vec64<i64> is aligned: {}", v64_aligned);
412 println!(
413 "Minarrow IntegerArray<i64> is aligned: {}",
414 int_array_aligned
415 );
416 println!("Arrow ArrowI64Array is aligned: {}", i64_arrow_aligned);
417 println!(
418 "Minarrow Array::NumericArray<i64> is aligned: {}",
419 arr_int_enum_aligned
420 );
421 println!("Arrow ArrayRef<int> is aligned: {}", array_ref_int_aligned);
422 println!("\n");
423 println!("Verify SIMD pointer alignment for Float calculations (based on lane width):");
424 println!("Vec<f64> is aligned: {}", v_float_aligned);
425 println!("Vec64<f64> is aligned: {}", v64_float_aligned);
426 println!("FloatArray<f64> is aligned: {}", float_arr_aligned);
427 println!("ArrowF64Array is aligned: {}", arrow_f64_aligned);
428 println!(
429 "Array::NumericArray<f64> is aligned: {}",
430 float_enum_aligned
431 );
432 println!("ArrayRef is aligned: {}", arrow_f64_arr_aligned);
433 println!("\n");
434
435 println!("---------------------- END OF SIMD BENCHMARKS ---------------------------");
436 }examples/benchmarks/hotloop_benchmark_avg_simd.rs (line 268)
182 pub fn run_benchmark(n: usize, simd_lanes: usize) {
183 let mut total_vec = std::time::Duration::ZERO;
184 let mut total_vec64 = std::time::Duration::ZERO;
185 let mut total_minarrow_direct = std::time::Duration::ZERO;
186 let mut total_arrow_struct = std::time::Duration::ZERO;
187 let mut total_minarrow_enum = std::time::Duration::ZERO;
188 let mut total_arrow_dyn = std::time::Duration::ZERO;
189
190 let mut total_vec_f64 = std::time::Duration::ZERO;
191 let mut total_vec64_f64 = std::time::Duration::ZERO;
192 let mut total_minarrow_direct_f64 = std::time::Duration::ZERO;
193 let mut total_arrow_struct_f64 = std::time::Duration::ZERO;
194 let mut total_minarrow_enum_f64 = std::time::Duration::ZERO;
195 let mut total_arrow_dyn_f64 = std::time::Duration::ZERO;
196
197 // Data construction - This is the only part we
198 // exclude from the overall benchmark, however, we time Vec
199 // vs. Vec64 here as an indicative profile, given this is the
200 // starting setup of all other reference points.
201 let mut sum_vec_i64 = 0u128;
202 let mut sum_vec64_i64 = 0u128;
203
204 // for keeping scope alive
205 // after the Vec benchmarks, we keep the last one each
206 let mut v_int_data = Vec::with_capacity(n);
207 let mut v64_int_data = Vec64::with_capacity(n);
208
209 for _ in 0..ITERATIONS {
210 let t0 = Instant::now();
211 v_int_data = (0..n as i64).collect();
212 let dur_vec_i64 = t0.elapsed();
213
214 let t1 = Instant::now();
215 v64_int_data = (0..n as i64).collect();
216 let dur_vec64_i64 = t1.elapsed();
217
218 sum_vec_i64 += dur_vec_i64.as_nanos();
219 sum_vec64_i64 += dur_vec64_i64.as_nanos();
220 }
221
222 let avg_vec_i64 = sum_vec_i64 as f64 / ITERATIONS as f64;
223 let avg_vec64_i64 = sum_vec64_i64 as f64 / ITERATIONS as f64;
224
225 println!(
226 "Vec<i64> construction (avg): {}",
227 fmt_duration_ns(avg_vec_i64)
228 );
229 println!(
230 "Vec64<i64> construction (avg): {}",
231 fmt_duration_ns(avg_vec64_i64)
232 );
233 println!("\n=> Keep the above Vec construction delta in mind when interpreting the below results,
234 as it is not included in the benchmarks that follow.\n");
235
236 // Alignment checks - once, outside timing
237
238 let v_aligned = {
239 (&v_int_data[0] as *const i64 as usize) % std::mem::align_of::<Simd<i64, SIMD_LANES>>()
240 == 0
241 };
242
243 let v64_aligned = {
244 (&v64_int_data[0] as *const i64 as usize)
245 % std::mem::align_of::<Simd<i64, SIMD_LANES>>()
246 == 0
247 };
248
249 let int_array_aligned = {
250 let int_arr = IntegerArray {
251 data: Buffer::from(v64_int_data.clone()),
252 null_mask: None,
253 };
254 let slice = &int_arr[..];
255 (slice.as_ptr() as usize) % std::mem::align_of::<Simd<i64, SIMD_LANES>>() == 0
256 };
257
258 let i64_arrow_aligned = {
259 let arr = ArrowI64Array::from(v_int_data.clone());
260 (arr.values().as_ptr() as usize) % std::mem::align_of::<Simd<i64, SIMD_LANES>>() == 0
261 };
262
263 let arr_int_enum_aligned = {
264 let array = Array::NumericArray(NumericArray::Int64(Arc::new(IntegerArray {
265 data: Buffer::from(v64_int_data.clone()),
266 null_mask: None,
267 })));
268 let int_arr = array.num().i64().unwrap();
269 (int_arr.data.as_slice().as_ptr() as usize)
270 % std::mem::align_of::<Simd<i64, SIMD_LANES>>()
271 == 0
272 };
273
274 let array_ref_int_aligned = {
275 let arr: ArrayRef = Arc::new(ArrowI64Array::from(v_int_data.clone()));
276 let int_arr = arr.as_any().downcast_ref::<ArrowI64Array>().unwrap();
277 (int_arr.values().as_ptr() as usize) % std::mem::align_of::<Simd<i64, SIMD_LANES>>()
278 == 0
279 };
280
281 let v_float_data: Vec<f64> = (0..n as i64).map(|x| x as f64).collect();
282 let v64_float_data: Vec64<f64> = (0..n as i64).map(|x| x as f64).collect();
283
284 let v_float_aligned = {
285 (&v_float_data[0] as *const f64 as usize)
286 % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
287 == 0
288 };
289
290 let v64_float_aligned = {
291 (&v64_float_data[0] as *const f64 as usize)
292 % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
293 == 0
294 };
295
296 let float_arr_aligned = {
297 let float_arr = FloatArray {
298 data: Buffer::from(v64_float_data.clone()),
299 null_mask: None,
300 };
301 (&float_arr.data.as_slice()[0] as *const f64 as usize)
302 % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
303 == 0
304 };
305
306 let arrow_f64_aligned = {
307 let arr = ArrowF64Array::from(v_float_data.clone());
308 (arr.values().as_ptr() as usize) % std::mem::align_of::<Simd<f64, SIMD_LANES>>() == 0
309 };
310
311 let float_enum_aligned = {
312 let array = Array::NumericArray(NumericArray::Float64(Arc::new(FloatArray {
313 data: Buffer::from(v64_float_data.clone()),
314 null_mask: None,
315 })));
316 let float_arr = array.num().f64().unwrap();
317 (float_arr.data.as_slice().as_ptr() as usize)
318 % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
319 == 0
320 };
321
322 let arrow_f64_arr_aligned = {
323 let arr: ArrayRef = Arc::new(ArrowF64Array::from(v_float_data.clone()));
324 let float_arr = arr.as_any().downcast_ref::<ArrowF64Array>().unwrap();
325 (float_arr.values().as_ptr() as usize) % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
326 == 0
327 };
328
329 for _ in 0..ITERATIONS {
330 // --- Integer (i64) tests ---
331 // Raw Vec<i64>
332 let data = v_int_data.clone();
333 let start = Instant::now();
334 let sum = simd_sum_i64_runtime(&data[..], simd_lanes);
335 let dur = start.elapsed();
336 total_vec += dur;
337 black_box(sum);
338
339 // Raw Vec64<i64>
340 let data: Vec64<i64> = v64_int_data.clone();
341 let start = Instant::now();
342 let sum = simd_sum_i64_runtime(&data[..], simd_lanes);
343 let dur = start.elapsed();
344 total_vec64 += dur;
345 black_box(sum);
346
347 // Minarrow i64 (direct struct)
348 let data: Vec64<i64> = v64_int_data.clone();
349 let start = Instant::now();
350 let int_arr = IntegerArray {
351 data: Buffer::from(data),
352 null_mask: None,
353 };
354 let sum = simd_sum_i64_runtime(&int_arr[..], simd_lanes);
355 let dur = start.elapsed();
356 total_minarrow_direct += dur;
357 black_box(sum);
358
359 // Arrow i64 (struct direct)
360 let data: Vec<i64> = v_int_data.clone();
361 let start = Instant::now();
362 let arr = ArrowI64Array::from(data);
363 let sum = simd_sum_i64_runtime(arr.values(), simd_lanes);
364 let dur = start.elapsed();
365 total_arrow_struct += dur;
366 black_box(sum);
367
368 // Minarrow i64 (enum)
369 let data: Vec64<i64> = v64_int_data.clone();
370 let start = Instant::now();
371 let array = Array::NumericArray(NumericArray::Int64(Arc::new(IntegerArray {
372 data: Buffer::from(data),
373 null_mask: None,
374 })));
375 let int_arr = array.num().i64().unwrap();
376 let sum = simd_sum_i64_runtime(&int_arr[..], simd_lanes);
377 let dur = start.elapsed();
378 total_minarrow_enum += dur;
379 black_box(sum);
380
381 // Arrow i64 (dynamic)
382 let data: Vec<i64> = v_int_data.clone();
383 let start = Instant::now();
384 let arr: ArrayRef = Arc::new(ArrowI64Array::from(data));
385 let int_arr = arr.as_any().downcast_ref::<ArrowI64Array>().unwrap();
386 let sum = simd_sum_i64_runtime(int_arr.values(), simd_lanes);
387 let dur = start.elapsed();
388 total_arrow_dyn += dur;
389 black_box(sum);
390
391 // --- Float (f64) tests ---
392
393 // Raw Vec<f64>
394 let data: Vec<f64> = v_float_data.clone();
395 let start = Instant::now();
396 let sum = simd_sum_f64_runtime(&data[..], simd_lanes);
397 let dur = start.elapsed();
398 total_vec_f64 += dur;
399 black_box(sum);
400
401 // Raw Vec64<f64>
402 let data: Vec64<f64> = v64_float_data.clone();
403 let start = Instant::now();
404 let sum = simd_sum_f64_runtime(&data[..], simd_lanes);
405 let dur = start.elapsed();
406 total_vec64_f64 += dur;
407 black_box(sum);
408
409 // Minarrow f64 (direct struct)
410 let data: Vec64<f64> = v64_float_data.clone();
411 let start = Instant::now();
412 let float_arr = FloatArray {
413 data: Buffer::from(data),
414 null_mask: None,
415 };
416 let sum = simd_sum_f64_runtime(&float_arr[..], simd_lanes);
417 let dur = start.elapsed();
418 total_minarrow_direct_f64 += dur;
419 black_box(sum);
420
421 // Arrow f64 (struct direct)
422 let data: Vec<f64> = v_float_data.clone();
423 let start = Instant::now();
424 let arr = ArrowF64Array::from(data);
425 let sum = simd_sum_f64_runtime(arr.values(), simd_lanes);
426 let dur = start.elapsed();
427 total_arrow_struct_f64 += dur;
428 black_box(sum);
429
430 // Minarrow f64 (enum)
431 let data: Vec64<f64> = v64_float_data.clone();
432 let start = Instant::now();
433 let array = Array::NumericArray(NumericArray::Float64(Arc::new(FloatArray {
434 data: Buffer::from(data),
435 null_mask: None,
436 })));
437 let float_arr = array.num().f64().unwrap();
438 let sum = simd_sum_f64_runtime(&float_arr[..], simd_lanes);
439 let dur = start.elapsed();
440 total_minarrow_enum_f64 += dur;
441 black_box(sum);
442
443 // Arrow f64 (dynamic)
444 let data: Vec<f64> = v_float_data.clone();
445 let start = Instant::now();
446 let arr: ArrayRef = Arc::new(ArrowF64Array::from(data));
447 let float_arr = arr.as_any().downcast_ref::<ArrowF64Array>().unwrap();
448 let sum = simd_sum_f64_runtime(float_arr.values(), simd_lanes);
449 let dur = start.elapsed();
450 total_arrow_dyn_f64 += dur;
451 black_box(sum);
452 }
453
454 println!("Averaged Results from {} runs:", ITERATIONS);
455 println!("---------------------------------");
456
457 let avg_vec = total_vec.as_nanos() as f64 / ITERATIONS as f64;
458 let avg_vec64 = total_vec64.as_nanos() as f64 / ITERATIONS as f64;
459 let avg_minarrow_direct = total_minarrow_direct.as_nanos() as f64 / ITERATIONS as f64;
460 let avg_arrow_struct = total_arrow_struct.as_nanos() as f64 / ITERATIONS as f64;
461 let avg_minarrow_enum = total_minarrow_enum.as_nanos() as f64 / ITERATIONS as f64;
462 let avg_arrow_dyn = total_arrow_dyn.as_nanos() as f64 / ITERATIONS as f64;
463
464 let avg_vec_f64 = total_vec_f64.as_nanos() as f64 / ITERATIONS as f64;
465 let avg_vec64_f64 = total_vec64_f64.as_nanos() as f64 / ITERATIONS as f64;
466 let avg_minarrow_direct_f64 =
467 total_minarrow_direct_f64.as_nanos() as f64 / ITERATIONS as f64;
468 let avg_arrow_struct_f64 = total_arrow_struct_f64.as_nanos() as f64 / ITERATIONS as f64;
469 let avg_minarrow_enum_f64 = total_minarrow_enum_f64.as_nanos() as f64 / ITERATIONS as f64;
470 let avg_arrow_dyn_f64 = total_arrow_dyn_f64.as_nanos() as f64 / ITERATIONS as f64;
471
472 println!("|------------ Integer Tests (SIMD) ------------|");
473 println!(
474 "raw vec: Vec<i64> avg = {} (n={})",
475 fmt_duration_ns(avg_vec),
476 ITERATIONS
477 );
478 println!(
479 "raw vec64: Vec64<i64> avg = {} (n={})",
480 fmt_duration_ns(avg_vec64),
481 ITERATIONS
482 );
483 println!(
484 "minarrow direct: IntegerArray avg = {} (n={})",
485 fmt_duration_ns(avg_minarrow_direct),
486 ITERATIONS
487 );
488 println!(
489 "arrow-rs struct: Int64Array avg = {} (n={})",
490 fmt_duration_ns(avg_arrow_struct),
491 ITERATIONS
492 );
493 println!(
494 "minarrow enum: IntegerArray avg = {} (n={})",
495 fmt_duration_ns(avg_minarrow_enum),
496 ITERATIONS
497 );
498 println!(
499 "arrow-rs dyn: Int64Array avg = {} (n={})",
500 fmt_duration_ns(avg_arrow_dyn),
501 ITERATIONS
502 );
503
504 println!();
505 println!("|------------ Float Tests (SIMD) --------------|");
506 println!(
507 "raw vec: Vec<f64> avg = {} (n={})",
508 fmt_duration_ns(avg_vec_f64),
509 ITERATIONS
510 );
511 println!(
512 "raw vec64: Vec64<f64> avg = {} (n={})",
513 fmt_duration_ns(avg_vec64_f64),
514 ITERATIONS
515 );
516 println!(
517 "minarrow direct: FloatArray avg = {} (n={})",
518 fmt_duration_ns(avg_minarrow_direct_f64),
519 ITERATIONS
520 );
521 println!(
522 "arrow-rs struct: Float64Array avg = {} (n={})",
523 fmt_duration_ns(avg_arrow_struct_f64),
524 ITERATIONS
525 );
526 println!(
527 "minarrow enum: FloatArray avg = {} (n={})",
528 fmt_duration_ns(avg_minarrow_enum_f64),
529 ITERATIONS
530 );
531 println!(
532 "arrow-rs dyn: Float64Array avg = {} (n={})",
533 fmt_duration_ns(avg_arrow_dyn_f64),
534 ITERATIONS
535 );
536
537 println!("\n=> Vec64 backs the above `Minarrow` types and `Vec` backs Arrow_Rs.");
538
539 println!("\nVerify SIMD pointer alignment for Integer calculations (based on lane width):");
540 println!("Vec<i64> is aligned: {}", v_aligned);
541 println!("Minarrow Vec64<i64> is aligned: {}", v64_aligned);
542 println!(
543 "Minarrow IntegerArray<i64> is aligned: {}",
544 int_array_aligned
545 );
546 println!("Arrow ArrowI64Array is aligned: {}", i64_arrow_aligned);
547 println!(
548 "Minarrow Array::NumericArray<i64> is aligned: {}",
549 arr_int_enum_aligned
550 );
551 println!("Arrow ArrayRef<int> is aligned: {}", array_ref_int_aligned);
552
553 println!("\nVerify SIMD pointer alignment for Float calculations (based on lane width):");
554 println!("Vec<f64> is aligned: {}", v_float_aligned);
555 println!("Vec64<f64> is aligned: {}", v64_float_aligned);
556 println!("FloatArray<f64> is aligned: {}", float_arr_aligned);
557 println!("ArrowF64Array is aligned: {}", arrow_f64_aligned);
558 println!(
559 "Array::NumericArray<f64> is aligned: {}",
560 float_enum_aligned
561 );
562 println!("ArrayRef is aligned: {}", arrow_f64_arr_aligned);
563
564 println!("\n---------------------- END OF SIMD AVG BENCHMARKS ---------------------------");
565 }Sourcepub fn u32(self) -> Result<IntegerArray<u32>, MinarrowError>
pub fn u32(self) -> Result<IntegerArray<u32>, MinarrowError>
Convert to IntegerArray
Sourcepub fn u64(self) -> Result<IntegerArray<u64>, MinarrowError>
pub fn u64(self) -> Result<IntegerArray<u64>, MinarrowError>
Convert to IntegerArray
Sourcepub fn f32(self) -> Result<FloatArray<f32>, MinarrowError>
pub fn f32(self) -> Result<FloatArray<f32>, MinarrowError>
Convert to FloatArray
Sourcepub fn f64(self) -> Result<FloatArray<f64>, MinarrowError>
pub fn f64(self) -> Result<FloatArray<f64>, MinarrowError>
Convert to FloatArray
Examples found in repository?
examples/benchmarks/hotloop_benchmark_std.rs (line 201)
33 pub(crate) fn run_benchmark() {
34 // ----------- Raw Vec<i64> -----------
35 let raw_vec: Vec<i64> = (0..N as i64).collect();
36 let start = Instant::now();
37 let mut acc = 0i64;
38 for &v in &raw_vec {
39 acc += v;
40 }
41 let dur_vec_i64 = start.elapsed();
42 println!("raw vec: Vec<i64> sum = {}, {:?}", acc, dur_vec_i64);
43 black_box(acc);
44 std::mem::drop(raw_vec);
45
46 // ----------- Raw Vec64<i64> -----------
47 let raw_vec: Vec64<i64> = (0..N as i64).collect();
48 let start = Instant::now();
49 let mut acc = 0i64;
50 for &v in &raw_vec {
51 acc += v;
52 }
53 let dur_vec_i64 = start.elapsed();
54 println!("raw vec: Vec64<i64> sum = {}, {:?}", acc, dur_vec_i64);
55 black_box(acc);
56 std::mem::drop(raw_vec);
57
58 // ----------- Minarrow i64 (direct struct, no enum) -----------
59 let min_data: Vec64<i64> = (0..N as i64).collect();
60 let start = Instant::now();
61 let int_arr = IntegerArray {
62 data: Buffer::from(min_data),
63 null_mask: None,
64 };
65 let mut acc = 0i64;
66 let slice = int_arr.data.as_slice();
67 for &v in slice {
68 acc += v;
69 }
70 let dur_minarrow_direct_i64 = start.elapsed();
71 println!(
72 "minarrow direct: IntegerArray sum = {}, {:?}",
73 acc, dur_minarrow_direct_i64
74 );
75 black_box(acc);
76 std::mem::drop(int_arr);
77
78 // ----------- Arrow i64 (struct direct) -----------
79 let data: Vec<i64> = (0..N as i64).collect();
80 let start = Instant::now();
81 let arr = ArrowI64Array::from(data);
82 let mut acc = 0i64;
83 for i in 0..arr.len() {
84 acc += arr.value(i);
85 }
86 let dur_arrow_struct_i64 = start.elapsed();
87 println!(
88 "arrow-rs struct: Int64Array sum = {}, {:?}",
89 acc, dur_arrow_struct_i64
90 );
91 black_box(acc);
92 std::mem::drop(arr);
93
94 // ----------- Minarrow i64 (enum) -----------
95 let min_data: Vec64<i64> = (0..N as i64).collect();
96 let start = Instant::now();
97 let array = Array::NumericArray(NumericArray::Int64(Arc::new(IntegerArray {
98 data: Buffer::from(min_data),
99 null_mask: None,
100 })));
101 let mut acc = 0i64;
102 let int_arr = array.num().i64().unwrap();
103 let slice = int_arr.data.as_slice();
104 for &v in slice {
105 acc += v;
106 }
107 let dur_minarrow_enum_i64 = start.elapsed();
108 println!(
109 "minarrow enum: IntegerArray sum = {}, {:?}",
110 acc, dur_minarrow_enum_i64
111 );
112 black_box(acc);
113 std::mem::drop(int_arr);
114
115 // ----------- Arrow i64 (dynamic) -----------
116 let data_dyn: Vec<i64> = (0..N as i64).collect();
117 let start = Instant::now();
118 let arr_dyn: ArrayRef = Arc::new(ArrowI64Array::from(data_dyn));
119 let mut acc = 0i64;
120 if let Some(int) = arr_dyn.as_any().downcast_ref::<ArrowI64Array>() {
121 for i in 0..int.len() {
122 acc += int.value(i);
123 }
124 }
125 let dur_arrow_dyn_i64 = start.elapsed();
126 println!(
127 "arrow-rs dyn: ArrayRef Int64Array sum = {}, {:?}",
128 acc, dur_arrow_dyn_i64
129 );
130 black_box(acc);
131 std::mem::drop(arr_dyn);
132
133 // ----------- Raw Vec<f64> -----------
134 let raw_vec: Vec<f64> = (0..N as i64).map(|x| x as f64).collect();
135 let start = Instant::now();
136 let mut acc = 0.0f64;
137 for &v in &raw_vec {
138 acc += v;
139 }
140 let dur_vec_f64 = start.elapsed();
141 println!("raw vec: Vec<f64> sum = {}, {:?}", acc, dur_vec_f64);
142 black_box(acc);
143 std::mem::drop(raw_vec);
144
145 // ----------- Raw Vec64<f64> -----------
146 let raw_vec: Vec64<f64> = (0..N as i64).map(|x| x as f64).collect();
147 let start = Instant::now();
148 let mut acc = 0.0f64;
149 for &v in &raw_vec {
150 acc += v;
151 }
152 let dur_vec_f64 = start.elapsed();
153 println!("raw vec: Vec<f64> sum = {}, {:?}", acc, dur_vec_f64);
154 black_box(acc);
155 std::mem::drop(raw_vec);
156
157 // ----------- Minarrow f64 (direct struct, no enum) -----------
158 let min_data_f64: Vec64<f64> = (0..N as i64).map(|x| x as f64).collect();
159 let start = Instant::now();
160 let float_arr = FloatArray {
161 data: Buffer::from(min_data_f64),
162 null_mask: None,
163 };
164 let mut acc = 0.0f64;
165 let slice = float_arr.data.as_slice();
166 for &v in slice {
167 acc += v;
168 }
169 let dur_minarrow_direct_f64 = start.elapsed();
170 println!(
171 "minarrow direct: FloatArray sum = {}, {:?}",
172 acc, dur_minarrow_direct_f64
173 );
174 black_box(acc);
175 std::mem::drop(float_arr);
176
177 // ----------- Arrow f64 (struct direct) -----------
178 let data_f64: Vec<f64> = (0..N as i64).map(|x| x as f64).collect();
179 let start = Instant::now();
180 let arr = ArrowF64Array::from(data_f64);
181 let mut acc = 0.0f64;
182 for i in 0..arr.len() {
183 acc += arr.value(i);
184 }
185 let dur_arrow_struct_f64 = start.elapsed();
186 println!(
187 "arrow-rs struct: Float64Array sum = {}, {:?}",
188 acc, dur_arrow_struct_f64
189 );
190 black_box(acc);
191 std::mem::drop(arr);
192
193 // ----------- Minarrow f64 (enum) -----------
194 let min_data_f64: Vec64<f64> = (0..N as i64).map(|x| x as f64).collect();
195 let start = Instant::now();
196 let array = Array::NumericArray(NumericArray::Float64(Arc::new(FloatArray {
197 data: Buffer::from(min_data_f64),
198 null_mask: None,
199 })));
200 let mut acc = 0.0f64;
201 let float_arr = array.num().f64().unwrap();
202 let slice = float_arr.data.as_slice();
203 for &v in slice {
204 acc += v;
205 }
206 let dur_minarrow_enum_f64 = start.elapsed();
207 println!(
208 "minarrow enum: FloatArray sum = {}, {:?}",
209 acc, dur_minarrow_enum_f64
210 );
211 black_box(acc);
212 std::mem::drop(float_arr);
213
214 // ----------- Arrow f64 (dynamic) -----------
215 let data_f64: Vec<f64> = (0..N as i64).map(|x| x as f64).collect();
216 let start = Instant::now();
217 let arr: ArrayRef = Arc::new(ArrowF64Array::from(data_f64));
218 let mut acc = 0.0f64;
219 if let Some(f) = arr.as_any().downcast_ref::<ArrowF64Array>() {
220 for i in 0..f.len() {
221 acc += f.value(i);
222 }
223 }
224 let dur_arrow_dyn_f64 = start.elapsed();
225 println!(
226 "arrow-rs dyn: Float64Array sum = {}, {:?}",
227 acc, dur_arrow_dyn_f64
228 );
229 black_box(acc);
230 std::mem::drop(arr);
231 }More examples
examples/benchmarks/hotloop_benchmark_avg_std.rs (line 153)
36 pub(crate) fn run_benchmark() {
37 let mut total_arrow_dyn_i64 = std::time::Duration::ZERO;
38 let mut total_arrow_struct_i64 = std::time::Duration::ZERO;
39 let mut total_minarrow_enum_i64 = std::time::Duration::ZERO;
40 let mut total_minarrow_direct_i64 = std::time::Duration::ZERO;
41 let mut total_vec_i64 = std::time::Duration::ZERO;
42 let mut total_arrow_dyn_f64 = std::time::Duration::ZERO;
43 let mut total_arrow_struct_f64 = std::time::Duration::ZERO;
44 let mut total_minarrow_enum_f64 = std::time::Duration::ZERO;
45 let mut total_minarrow_direct_f64 = std::time::Duration::ZERO;
46 let mut total_vec_f64 = std::time::Duration::ZERO;
47
48 for _ in 0..ITERATIONS {
49 // ----------- Arrow i64 (dynamic) -----------
50 let data: Vec<i64> = (0..N as i64).collect();
51 let start = Instant::now();
52 let arr: ArrayRef = Arc::new(ArrowI64Array::from(data));
53 let mut acc = 0i64;
54 if let Some(int) = arr.as_any().downcast_ref::<ArrowI64Array>() {
55 for i in 0..int.len() {
56 acc += int.value(i);
57 }
58 }
59 let dur_arrow_dyn_i64 = start.elapsed();
60 total_arrow_dyn_i64 += dur_arrow_dyn_i64;
61 black_box(acc);
62
63 // ----------- Arrow i64 (struct direct) -----------
64 let data: Vec<i64> = (0..N as i64).collect();
65 let start = Instant::now();
66 let arr = ArrowI64Array::from(data);
67 let mut acc = 0i64;
68 for i in 0..arr.len() {
69 acc += arr.value(i);
70 }
71 let dur_arrow_struct_i64 = start.elapsed();
72 total_arrow_struct_i64 += dur_arrow_struct_i64;
73 black_box(acc);
74
75 // ----------- Minarrow i64 (enum) -----------
76 let min_data: Vec64<i64> = (0..N as i64).collect();
77 let start = Instant::now();
78 let array = Array::NumericArray(NumericArray::Int64(Arc::new(IntegerArray {
79 data: Buffer::from(min_data),
80 null_mask: None,
81 })));
82 let mut acc = 0i64;
83 let int_arr = array.num().i64().unwrap();
84 let slice = int_arr.data.as_slice();
85 for &v in slice {
86 acc += v;
87 }
88 let dur_minarrow_enum_i64 = start.elapsed();
89 total_minarrow_enum_i64 += dur_minarrow_enum_i64;
90 black_box(acc);
91
92 // ----------- Minarrow i64 (direct struct, no enum) -----------
93 let min_data: Vec64<i64> = (0..N as i64).collect();
94 let start = Instant::now();
95 let int_arr = IntegerArray {
96 data: Buffer::from(min_data),
97 null_mask: None,
98 };
99 let mut acc = 0i64;
100 let slice = int_arr.data.as_slice();
101 for &v in slice {
102 acc += v;
103 }
104 let dur_minarrow_direct_i64 = start.elapsed();
105 total_minarrow_direct_i64 += dur_minarrow_direct_i64;
106 black_box(acc);
107
108 // ----------- Raw Vec<i64> -----------
109 let raw_vec: Vec<i64> = (0..N as i64).collect();
110 let start = Instant::now();
111 let mut acc = 0i64;
112 for &v in &raw_vec {
113 acc += v;
114 }
115 let dur_vec_i64 = start.elapsed();
116 total_vec_i64 += dur_vec_i64;
117 black_box(acc);
118
119 // ----------- Arrow f64 (dynamic) -----------
120 let data_f64: Vec<f64> = (0..N as i64).map(|x| x as f64).collect();
121 let start = Instant::now();
122 let arr: ArrayRef = Arc::new(ArrowF64Array::from(data_f64));
123 let mut acc = 0.0f64;
124 if let Some(f) = arr.as_any().downcast_ref::<ArrowF64Array>() {
125 for i in 0..f.len() {
126 acc += f.value(i);
127 }
128 }
129 let dur_arrow_dyn_f64 = start.elapsed();
130 total_arrow_dyn_f64 += dur_arrow_dyn_f64;
131 black_box(acc);
132
133 // ----------- Arrow f64 (struct direct) -----------
134 let data_f64: Vec<f64> = (0..N as i64).map(|x| x as f64).collect();
135 let start = Instant::now();
136 let arr = ArrowF64Array::from(data_f64);
137 let mut acc = 0.0f64;
138 for i in 0..arr.len() {
139 acc += arr.value(i);
140 }
141 let dur_arrow_struct_f64 = start.elapsed();
142 total_arrow_struct_f64 += dur_arrow_struct_f64;
143 black_box(acc);
144
145 // ----------- Minarrow f64 (enum) -----------
146 let min_data_f64: Vec64<f64> = (0..N as i64).map(|x| x as f64).collect();
147 let start = Instant::now();
148 let array = Array::NumericArray(NumericArray::Float64(Arc::new(FloatArray {
149 data: Buffer::from(min_data_f64),
150 null_mask: None,
151 })));
152 let mut acc = 0.0f64;
153 let float_arr = array.num().f64().unwrap();
154 let slice = float_arr.data.as_slice();
155 for &v in slice {
156 acc += v;
157 }
158 let dur_minarrow_enum_f64 = start.elapsed();
159 total_minarrow_enum_f64 += dur_minarrow_enum_f64;
160 black_box(acc);
161
162 // ----------- Minarrow f64 (direct struct, no enum) -----------
163 let min_data_f64: Vec64<f64> = (0..N as i64).map(|x| x as f64).collect();
164 let start = Instant::now();
165 let float_arr = FloatArray {
166 data: Buffer::from(min_data_f64),
167 null_mask: None,
168 };
169 let mut acc = 0.0f64;
170 let slice = float_arr.data.as_slice();
171 for &v in slice {
172 acc += v;
173 }
174 let dur_minarrow_direct_f64 = start.elapsed();
175 total_minarrow_direct_f64 += dur_minarrow_direct_f64;
176 black_box(acc);
177
178 // ----------- Raw Vec<f64> -----------
179 let raw_vec: Vec<f64> = (0..N as i64).map(|x| x as f64).collect();
180 let start = Instant::now();
181 let mut acc = 0.0f64;
182 for &v in &raw_vec {
183 acc += v;
184 }
185 let dur_vec_f64 = start.elapsed();
186 total_vec_f64 += dur_vec_f64;
187 black_box(acc);
188 }
189
190 println!("Averaged Results from {} runs:", ITERATIONS);
191 println!("---------------------------------");
192
193 let avg_vec_i64 = total_vec_i64.as_nanos() as f64 / ITERATIONS as f64;
194 let avg_minarrow_direct_i64 =
195 total_minarrow_direct_i64.as_nanos() as f64 / ITERATIONS as f64;
196 let avg_arrow_struct_i64 = total_arrow_struct_i64.as_nanos() as f64 / ITERATIONS as f64;
197 let avg_minarrow_enum_i64 = total_minarrow_enum_i64.as_nanos() as f64 / ITERATIONS as f64;
198 let avg_arrow_dyn_i64 = total_arrow_dyn_i64.as_nanos() as f64 / ITERATIONS as f64;
199
200 let avg_vec_f64 = total_vec_f64.as_nanos() as f64 / ITERATIONS as f64;
201 let avg_minarrow_direct_f64 =
202 total_minarrow_direct_f64.as_nanos() as f64 / ITERATIONS as f64;
203 let avg_arrow_struct_f64 = total_arrow_struct_f64.as_nanos() as f64 / ITERATIONS as f64;
204 let avg_minarrow_enum_f64 = total_minarrow_enum_f64.as_nanos() as f64 / ITERATIONS as f64;
205 let avg_arrow_dyn_f64 = total_arrow_dyn_f64.as_nanos() as f64 / ITERATIONS as f64;
206
207 println!(
208 "raw vec: Vec<i64> avg = {} (n={})",
209 fmt_duration_ns(avg_vec_i64),
210 ITERATIONS
211 );
212 println!(
213 "minarrow direct: IntegerArray avg = {} (n={})",
214 fmt_duration_ns(avg_minarrow_direct_i64),
215 ITERATIONS
216 );
217 println!(
218 "arrow-rs struct: Int64Array avg = {} (n={})",
219 fmt_duration_ns(avg_arrow_struct_i64),
220 ITERATIONS
221 );
222 println!();
223 println!(
224 "minarrow enum: IntegerArray avg = {} (n={})",
225 fmt_duration_ns(avg_minarrow_enum_i64),
226 ITERATIONS
227 );
228 println!(
229 "arrow-rs dyn: Int64Array avg = {} (n={})",
230 fmt_duration_ns(avg_arrow_dyn_i64),
231 ITERATIONS
232 );
233 println!();
234 println!(
235 "raw vec: Vec<f64> avg = {} (n={})",
236 fmt_duration_ns(avg_vec_f64),
237 ITERATIONS
238 );
239 println!(
240 "minarrow direct: FloatArray avg = {} (n={})",
241 fmt_duration_ns(avg_minarrow_direct_f64),
242 ITERATIONS
243 );
244 println!(
245 "arrow-rs struct: Float64Array avg = {} (n={})",
246 fmt_duration_ns(avg_arrow_struct_f64),
247 ITERATIONS
248 );
249 println!();
250 println!(
251 "minarrow enum: FloatArray avg = {} (n={})",
252 fmt_duration_ns(avg_minarrow_enum_f64),
253 ITERATIONS
254 );
255 println!(
256 "arrow-rs dyn: Float64Array avg = {} (n={})",
257 fmt_duration_ns(avg_arrow_dyn_f64),
258 ITERATIONS
259 );
260 }examples/benchmarks/hotloop_benchmark_simd.rs (line 375)
188 pub fn run_benchmark(n: usize, simd_lanes: usize) {
189 // ----------- Integer (i64) tests -----------
190
191 let data: Vec<i64> = (0..n as i64).collect();
192 black_box(simd_sum_i64_runtime(&data[..], simd_lanes)); // warmup, ignore result
193
194 println!("|------------ Integer Tests ------------ |\n");
195 // Raw Vec<i64>
196 // Sometimes this will randomly align, other times it will not.
197 let data: Vec<i64> = (0..n as i64).collect();
198 let start = Instant::now();
199 let slice = &data[..];
200 let sum = simd_sum_i64_runtime(slice, simd_lanes);
201 let dur_vec = start.elapsed();
202 println!("raw vec: Vec<i64> sum = {}, {:?}", sum, dur_vec);
203 let v_aligned =
204 (&data[0] as *const i64 as usize) % std::mem::align_of::<Simd<i64, SIMD_LANES>>() == 0;
205 black_box(sum);
206
207 // Raw Vec64<i64>
208 let data: Vec64<i64> = (0..n as i64).collect();
209 let start = Instant::now();
210 let slice = &data[..];
211 let sum = simd_sum_i64_runtime(slice, simd_lanes);
212 let dur_vec = start.elapsed();
213 println!("raw vec64: Vec64<i64> sum = {}, {:?}", sum, dur_vec);
214 let v64_aligned =
215 (&data[0] as *const i64 as usize) % std::mem::align_of::<Simd<i64, SIMD_LANES>>() == 0;
216 black_box(sum);
217
218 // Minarrow i64 (direct struct)
219 let data: Vec64<i64> = (0..n as i64).collect();
220 let data_copy = data.clone();
221
222 let start = Instant::now();
223 let int_arr = IntegerArray {
224 data: Buffer::from(data),
225 null_mask: None,
226 };
227 let slice = &int_arr[..];
228 let sum = simd_sum_i64_runtime(slice, simd_lanes);
229 let dur_minarrow_direct = start.elapsed();
230 println!(
231 "minarrow direct: IntegerArray sum = {}, {:?}",
232 sum, dur_minarrow_direct
233 );
234 let int_array_aligned = (&data_copy[0] as *const i64 as usize)
235 % std::mem::align_of::<Simd<i64, SIMD_LANES>>()
236 == 0;
237 black_box(sum);
238
239 // Arrow i64 (struct direct)
240 let data: Vec<i64> = (0..n as i64).collect();
241 let data_copy = data.clone();
242
243 let start = Instant::now();
244 let arr = ArrowI64Array::from(data);
245 let slice = arr.values();
246 let sum = simd_sum_i64_runtime(slice, simd_lanes);
247 let dur_arrow_struct = start.elapsed();
248 println!(
249 "arrow-rs struct: Int64Array sum = {}, {:?}",
250 sum, dur_arrow_struct
251 );
252 let i64_arrow_aligned = (&data_copy[0] as *const i64 as usize)
253 % std::mem::align_of::<Simd<i64, SIMD_LANES>>()
254 == 0;
255 black_box(sum);
256
257 // Minarrow i64 (enum)
258 let data: Vec64<i64> = (0..n as i64).collect();
259 let data_copy = data.clone();
260
261 let start = Instant::now();
262 let array = Array::NumericArray(NumericArray::Int64(Arc::new(IntegerArray {
263 data: Buffer::from(data),
264 null_mask: None,
265 })));
266 let int_arr = array.num().i64().unwrap();
267 let slice = &int_arr[..];
268 let sum = simd_sum_i64_runtime(slice, simd_lanes);
269 let dur_minarrow_enum = start.elapsed();
270 println!(
271 "minarrow enum: IntegerArray sum = {}, {:?}",
272 sum, dur_minarrow_enum
273 );
274 let arr_int_enum_aligned = (&data_copy[0] as *const i64 as usize)
275 % std::mem::align_of::<Simd<i64, SIMD_LANES>>()
276 == 0;
277 black_box(sum);
278
279 // Arrow i64 (dynamic)
280 let data: Vec<i64> = (0..n as i64).collect();
281 let data_copy = data.clone();
282
283 let start = Instant::now();
284 let arr: ArrayRef = Arc::new(ArrowI64Array::from(data));
285 let slice = if let Some(f) = arr.as_any().downcast_ref::<ArrowI64Array>() {
286 f.values()
287 } else {
288 panic!("downcast failed")
289 };
290 let sum = simd_sum_i64_runtime(slice, simd_lanes);
291 let dur_arrow_dyn_i64 = start.elapsed();
292 println!(
293 "arrow-rs dyn: Int64Array sum = {}, {:?}",
294 sum, dur_arrow_dyn_i64
295 );
296 let array_ref_int_aligned = (&data_copy[0] as *const i64 as usize)
297 % std::mem::align_of::<Simd<i64, SIMD_LANES>>()
298 == 0;
299 black_box(sum);
300 println!("\n");
301
302 // ----------- Float (f64) tests ----------------
303
304 println!("|------------ Float Tests ------------ |\n");
305
306 // Raw Vec<f64>
307 // Sometimes this will randomly align, other times it will not.
308 let data: Vec<f64> = (0..n as i64).map(|x| x as f64).collect();
309 let start = Instant::now();
310 let sum = simd_sum_f64_runtime(&data[..], simd_lanes);
311 let dur_vec_f64 = start.elapsed();
312 println!("raw vec: Vec<f64> sum = {}, {:?}", sum, dur_vec_f64);
313 let v_float_aligned =
314 (&data[0] as *const f64 as usize) % std::mem::align_of::<Simd<f64, SIMD_LANES>>() == 0;
315
316 black_box(sum);
317
318 // Raw Vec64<f64>
319 let data: Vec64<f64> = (0..n as i64).map(|x| x as f64).collect();
320 let start = Instant::now();
321 let sum = simd_sum_f64_runtime(&data[..], simd_lanes);
322 let dur_vec_f64 = start.elapsed();
323 println!("raw vec64: Vec64<f64> sum = {}, {:?}", sum, dur_vec_f64);
324 let v64_float_aligned =
325 (&data[0] as *const f64 as usize) % std::mem::align_of::<Simd<f64, SIMD_LANES>>() == 0;
326
327 black_box(sum);
328
329 // Minarrow f64 (direct struct, SIMD)
330 let data: Vec64<f64> = (0..n as i64).map(|x| x as f64).collect();
331 let data_copy = data.clone();
332
333 let start = Instant::now();
334 let float_arr = FloatArray {
335 data: Buffer::from(data),
336 null_mask: None,
337 };
338 let sum = simd_sum_f64_runtime(&float_arr[..], simd_lanes);
339 let dur_minarrow_direct_f64 = start.elapsed();
340 println!(
341 "minarrow direct: FloatArray sum = {}, {:?}",
342 sum, dur_minarrow_direct_f64
343 );
344 let float_arr_aligned = (&data_copy[0] as *const f64 as usize)
345 % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
346 == 0;
347 black_box(sum);
348
349 // Arrow f64 (struct direct)
350 let data: Vec<f64> = (0..n as i64).map(|x| x as f64).collect();
351 let data_copy = data.clone();
352
353 let start = Instant::now();
354 let arr = ArrowF64Array::from(data);
355 let sum = simd_sum_f64_runtime(arr.values(), simd_lanes);
356 let dur_arrow_struct_f64 = start.elapsed();
357 println!(
358 "arrow-rs struct: Float64Array sum = {}, {:?}",
359 sum, dur_arrow_struct_f64
360 );
361 let arrow_f64_aligned = (&data_copy[0] as *const f64 as usize)
362 % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
363 == 0;
364 black_box(sum);
365
366 // Minarrow f64 (enum)
367 let data: Vec64<f64> = (0..n as i64).map(|x| x as f64).collect();
368 let data_copy = data.clone();
369
370 let start = Instant::now();
371 let array = Array::NumericArray(NumericArray::Float64(Arc::new(FloatArray {
372 data: Buffer::from(data),
373 null_mask: None,
374 })));
375 let float_arr = array.num().f64().unwrap();
376 let sum = simd_sum_f64_runtime(&float_arr[..], simd_lanes);
377 let dur_minarrow_enum_f64 = start.elapsed();
378 println!(
379 "minarrow enum: FloatArray sum = {}, {:?}",
380 sum, dur_minarrow_enum_f64
381 );
382 let float_enum_aligned = (&data_copy[0] as *const f64 as usize)
383 % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
384 == 0;
385 black_box(sum);
386
387 // Arrow f64 (dynamic)
388 let data: Vec<f64> = (0..n as i64).map(|x| x as f64).collect();
389 let data_copy = data.clone();
390
391 let start = Instant::now();
392 let arr: ArrayRef = Arc::new(ArrowF64Array::from(data));
393 let slice = if let Some(f) = arr.as_any().downcast_ref::<ArrowF64Array>() {
394 f.values()
395 } else {
396 panic!("downcast failed")
397 };
398 let sum = simd_sum_f64_runtime(slice, simd_lanes);
399 let dur_arrow_dyn_f64 = start.elapsed();
400 println!(
401 "arrow-rs dyn: Float64Array sum = {}, {:?}",
402 sum, dur_arrow_dyn_f64
403 );
404 let arrow_f64_arr_aligned = (&data_copy[0] as *const f64 as usize)
405 % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
406 == 0;
407 black_box(sum);
408 println!("\n");
409 println!("Verify SIMD pointer alignment for Integer calculations (based on lane width):");
410 println!("Vec<i64> is aligned: {}", v_aligned);
411 println!("Minarrow Vec64<i64> is aligned: {}", v64_aligned);
412 println!(
413 "Minarrow IntegerArray<i64> is aligned: {}",
414 int_array_aligned
415 );
416 println!("Arrow ArrowI64Array is aligned: {}", i64_arrow_aligned);
417 println!(
418 "Minarrow Array::NumericArray<i64> is aligned: {}",
419 arr_int_enum_aligned
420 );
421 println!("Arrow ArrayRef<int> is aligned: {}", array_ref_int_aligned);
422 println!("\n");
423 println!("Verify SIMD pointer alignment for Float calculations (based on lane width):");
424 println!("Vec<f64> is aligned: {}", v_float_aligned);
425 println!("Vec64<f64> is aligned: {}", v64_float_aligned);
426 println!("FloatArray<f64> is aligned: {}", float_arr_aligned);
427 println!("ArrowF64Array is aligned: {}", arrow_f64_aligned);
428 println!(
429 "Array::NumericArray<f64> is aligned: {}",
430 float_enum_aligned
431 );
432 println!("ArrayRef is aligned: {}", arrow_f64_arr_aligned);
433 println!("\n");
434
435 println!("---------------------- END OF SIMD BENCHMARKS ---------------------------");
436 }examples/benchmarks/hotloop_benchmark_avg_simd.rs (line 316)
182 pub fn run_benchmark(n: usize, simd_lanes: usize) {
183 let mut total_vec = std::time::Duration::ZERO;
184 let mut total_vec64 = std::time::Duration::ZERO;
185 let mut total_minarrow_direct = std::time::Duration::ZERO;
186 let mut total_arrow_struct = std::time::Duration::ZERO;
187 let mut total_minarrow_enum = std::time::Duration::ZERO;
188 let mut total_arrow_dyn = std::time::Duration::ZERO;
189
190 let mut total_vec_f64 = std::time::Duration::ZERO;
191 let mut total_vec64_f64 = std::time::Duration::ZERO;
192 let mut total_minarrow_direct_f64 = std::time::Duration::ZERO;
193 let mut total_arrow_struct_f64 = std::time::Duration::ZERO;
194 let mut total_minarrow_enum_f64 = std::time::Duration::ZERO;
195 let mut total_arrow_dyn_f64 = std::time::Duration::ZERO;
196
197 // Data construction - This is the only part we
198 // exclude from the overall benchmark, however, we time Vec
199 // vs. Vec64 here as an indicative profile, given this is the
200 // starting setup of all other reference points.
201 let mut sum_vec_i64 = 0u128;
202 let mut sum_vec64_i64 = 0u128;
203
204 // for keeping scope alive
205 // after the Vec benchmarks, we keep the last one each
206 let mut v_int_data = Vec::with_capacity(n);
207 let mut v64_int_data = Vec64::with_capacity(n);
208
209 for _ in 0..ITERATIONS {
210 let t0 = Instant::now();
211 v_int_data = (0..n as i64).collect();
212 let dur_vec_i64 = t0.elapsed();
213
214 let t1 = Instant::now();
215 v64_int_data = (0..n as i64).collect();
216 let dur_vec64_i64 = t1.elapsed();
217
218 sum_vec_i64 += dur_vec_i64.as_nanos();
219 sum_vec64_i64 += dur_vec64_i64.as_nanos();
220 }
221
222 let avg_vec_i64 = sum_vec_i64 as f64 / ITERATIONS as f64;
223 let avg_vec64_i64 = sum_vec64_i64 as f64 / ITERATIONS as f64;
224
225 println!(
226 "Vec<i64> construction (avg): {}",
227 fmt_duration_ns(avg_vec_i64)
228 );
229 println!(
230 "Vec64<i64> construction (avg): {}",
231 fmt_duration_ns(avg_vec64_i64)
232 );
233 println!("\n=> Keep the above Vec construction delta in mind when interpreting the below results,
234 as it is not included in the benchmarks that follow.\n");
235
236 // Alignment checks - once, outside timing
237
238 let v_aligned = {
239 (&v_int_data[0] as *const i64 as usize) % std::mem::align_of::<Simd<i64, SIMD_LANES>>()
240 == 0
241 };
242
243 let v64_aligned = {
244 (&v64_int_data[0] as *const i64 as usize)
245 % std::mem::align_of::<Simd<i64, SIMD_LANES>>()
246 == 0
247 };
248
249 let int_array_aligned = {
250 let int_arr = IntegerArray {
251 data: Buffer::from(v64_int_data.clone()),
252 null_mask: None,
253 };
254 let slice = &int_arr[..];
255 (slice.as_ptr() as usize) % std::mem::align_of::<Simd<i64, SIMD_LANES>>() == 0
256 };
257
258 let i64_arrow_aligned = {
259 let arr = ArrowI64Array::from(v_int_data.clone());
260 (arr.values().as_ptr() as usize) % std::mem::align_of::<Simd<i64, SIMD_LANES>>() == 0
261 };
262
263 let arr_int_enum_aligned = {
264 let array = Array::NumericArray(NumericArray::Int64(Arc::new(IntegerArray {
265 data: Buffer::from(v64_int_data.clone()),
266 null_mask: None,
267 })));
268 let int_arr = array.num().i64().unwrap();
269 (int_arr.data.as_slice().as_ptr() as usize)
270 % std::mem::align_of::<Simd<i64, SIMD_LANES>>()
271 == 0
272 };
273
274 let array_ref_int_aligned = {
275 let arr: ArrayRef = Arc::new(ArrowI64Array::from(v_int_data.clone()));
276 let int_arr = arr.as_any().downcast_ref::<ArrowI64Array>().unwrap();
277 (int_arr.values().as_ptr() as usize) % std::mem::align_of::<Simd<i64, SIMD_LANES>>()
278 == 0
279 };
280
281 let v_float_data: Vec<f64> = (0..n as i64).map(|x| x as f64).collect();
282 let v64_float_data: Vec64<f64> = (0..n as i64).map(|x| x as f64).collect();
283
284 let v_float_aligned = {
285 (&v_float_data[0] as *const f64 as usize)
286 % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
287 == 0
288 };
289
290 let v64_float_aligned = {
291 (&v64_float_data[0] as *const f64 as usize)
292 % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
293 == 0
294 };
295
296 let float_arr_aligned = {
297 let float_arr = FloatArray {
298 data: Buffer::from(v64_float_data.clone()),
299 null_mask: None,
300 };
301 (&float_arr.data.as_slice()[0] as *const f64 as usize)
302 % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
303 == 0
304 };
305
306 let arrow_f64_aligned = {
307 let arr = ArrowF64Array::from(v_float_data.clone());
308 (arr.values().as_ptr() as usize) % std::mem::align_of::<Simd<f64, SIMD_LANES>>() == 0
309 };
310
311 let float_enum_aligned = {
312 let array = Array::NumericArray(NumericArray::Float64(Arc::new(FloatArray {
313 data: Buffer::from(v64_float_data.clone()),
314 null_mask: None,
315 })));
316 let float_arr = array.num().f64().unwrap();
317 (float_arr.data.as_slice().as_ptr() as usize)
318 % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
319 == 0
320 };
321
322 let arrow_f64_arr_aligned = {
323 let arr: ArrayRef = Arc::new(ArrowF64Array::from(v_float_data.clone()));
324 let float_arr = arr.as_any().downcast_ref::<ArrowF64Array>().unwrap();
325 (float_arr.values().as_ptr() as usize) % std::mem::align_of::<Simd<f64, SIMD_LANES>>()
326 == 0
327 };
328
329 for _ in 0..ITERATIONS {
330 // --- Integer (i64) tests ---
331 // Raw Vec<i64>
332 let data = v_int_data.clone();
333 let start = Instant::now();
334 let sum = simd_sum_i64_runtime(&data[..], simd_lanes);
335 let dur = start.elapsed();
336 total_vec += dur;
337 black_box(sum);
338
339 // Raw Vec64<i64>
340 let data: Vec64<i64> = v64_int_data.clone();
341 let start = Instant::now();
342 let sum = simd_sum_i64_runtime(&data[..], simd_lanes);
343 let dur = start.elapsed();
344 total_vec64 += dur;
345 black_box(sum);
346
347 // Minarrow i64 (direct struct)
348 let data: Vec64<i64> = v64_int_data.clone();
349 let start = Instant::now();
350 let int_arr = IntegerArray {
351 data: Buffer::from(data),
352 null_mask: None,
353 };
354 let sum = simd_sum_i64_runtime(&int_arr[..], simd_lanes);
355 let dur = start.elapsed();
356 total_minarrow_direct += dur;
357 black_box(sum);
358
359 // Arrow i64 (struct direct)
360 let data: Vec<i64> = v_int_data.clone();
361 let start = Instant::now();
362 let arr = ArrowI64Array::from(data);
363 let sum = simd_sum_i64_runtime(arr.values(), simd_lanes);
364 let dur = start.elapsed();
365 total_arrow_struct += dur;
366 black_box(sum);
367
368 // Minarrow i64 (enum)
369 let data: Vec64<i64> = v64_int_data.clone();
370 let start = Instant::now();
371 let array = Array::NumericArray(NumericArray::Int64(Arc::new(IntegerArray {
372 data: Buffer::from(data),
373 null_mask: None,
374 })));
375 let int_arr = array.num().i64().unwrap();
376 let sum = simd_sum_i64_runtime(&int_arr[..], simd_lanes);
377 let dur = start.elapsed();
378 total_minarrow_enum += dur;
379 black_box(sum);
380
381 // Arrow i64 (dynamic)
382 let data: Vec<i64> = v_int_data.clone();
383 let start = Instant::now();
384 let arr: ArrayRef = Arc::new(ArrowI64Array::from(data));
385 let int_arr = arr.as_any().downcast_ref::<ArrowI64Array>().unwrap();
386 let sum = simd_sum_i64_runtime(int_arr.values(), simd_lanes);
387 let dur = start.elapsed();
388 total_arrow_dyn += dur;
389 black_box(sum);
390
391 // --- Float (f64) tests ---
392
393 // Raw Vec<f64>
394 let data: Vec<f64> = v_float_data.clone();
395 let start = Instant::now();
396 let sum = simd_sum_f64_runtime(&data[..], simd_lanes);
397 let dur = start.elapsed();
398 total_vec_f64 += dur;
399 black_box(sum);
400
401 // Raw Vec64<f64>
402 let data: Vec64<f64> = v64_float_data.clone();
403 let start = Instant::now();
404 let sum = simd_sum_f64_runtime(&data[..], simd_lanes);
405 let dur = start.elapsed();
406 total_vec64_f64 += dur;
407 black_box(sum);
408
409 // Minarrow f64 (direct struct)
410 let data: Vec64<f64> = v64_float_data.clone();
411 let start = Instant::now();
412 let float_arr = FloatArray {
413 data: Buffer::from(data),
414 null_mask: None,
415 };
416 let sum = simd_sum_f64_runtime(&float_arr[..], simd_lanes);
417 let dur = start.elapsed();
418 total_minarrow_direct_f64 += dur;
419 black_box(sum);
420
421 // Arrow f64 (struct direct)
422 let data: Vec<f64> = v_float_data.clone();
423 let start = Instant::now();
424 let arr = ArrowF64Array::from(data);
425 let sum = simd_sum_f64_runtime(arr.values(), simd_lanes);
426 let dur = start.elapsed();
427 total_arrow_struct_f64 += dur;
428 black_box(sum);
429
430 // Minarrow f64 (enum)
431 let data: Vec64<f64> = v64_float_data.clone();
432 let start = Instant::now();
433 let array = Array::NumericArray(NumericArray::Float64(Arc::new(FloatArray {
434 data: Buffer::from(data),
435 null_mask: None,
436 })));
437 let float_arr = array.num().f64().unwrap();
438 let sum = simd_sum_f64_runtime(&float_arr[..], simd_lanes);
439 let dur = start.elapsed();
440 total_minarrow_enum_f64 += dur;
441 black_box(sum);
442
443 // Arrow f64 (dynamic)
444 let data: Vec<f64> = v_float_data.clone();
445 let start = Instant::now();
446 let arr: ArrayRef = Arc::new(ArrowF64Array::from(data));
447 let float_arr = arr.as_any().downcast_ref::<ArrowF64Array>().unwrap();
448 let sum = simd_sum_f64_runtime(float_arr.values(), simd_lanes);
449 let dur = start.elapsed();
450 total_arrow_dyn_f64 += dur;
451 black_box(sum);
452 }
453
454 println!("Averaged Results from {} runs:", ITERATIONS);
455 println!("---------------------------------");
456
457 let avg_vec = total_vec.as_nanos() as f64 / ITERATIONS as f64;
458 let avg_vec64 = total_vec64.as_nanos() as f64 / ITERATIONS as f64;
459 let avg_minarrow_direct = total_minarrow_direct.as_nanos() as f64 / ITERATIONS as f64;
460 let avg_arrow_struct = total_arrow_struct.as_nanos() as f64 / ITERATIONS as f64;
461 let avg_minarrow_enum = total_minarrow_enum.as_nanos() as f64 / ITERATIONS as f64;
462 let avg_arrow_dyn = total_arrow_dyn.as_nanos() as f64 / ITERATIONS as f64;
463
464 let avg_vec_f64 = total_vec_f64.as_nanos() as f64 / ITERATIONS as f64;
465 let avg_vec64_f64 = total_vec64_f64.as_nanos() as f64 / ITERATIONS as f64;
466 let avg_minarrow_direct_f64 =
467 total_minarrow_direct_f64.as_nanos() as f64 / ITERATIONS as f64;
468 let avg_arrow_struct_f64 = total_arrow_struct_f64.as_nanos() as f64 / ITERATIONS as f64;
469 let avg_minarrow_enum_f64 = total_minarrow_enum_f64.as_nanos() as f64 / ITERATIONS as f64;
470 let avg_arrow_dyn_f64 = total_arrow_dyn_f64.as_nanos() as f64 / ITERATIONS as f64;
471
472 println!("|------------ Integer Tests (SIMD) ------------|");
473 println!(
474 "raw vec: Vec<i64> avg = {} (n={})",
475 fmt_duration_ns(avg_vec),
476 ITERATIONS
477 );
478 println!(
479 "raw vec64: Vec64<i64> avg = {} (n={})",
480 fmt_duration_ns(avg_vec64),
481 ITERATIONS
482 );
483 println!(
484 "minarrow direct: IntegerArray avg = {} (n={})",
485 fmt_duration_ns(avg_minarrow_direct),
486 ITERATIONS
487 );
488 println!(
489 "arrow-rs struct: Int64Array avg = {} (n={})",
490 fmt_duration_ns(avg_arrow_struct),
491 ITERATIONS
492 );
493 println!(
494 "minarrow enum: IntegerArray avg = {} (n={})",
495 fmt_duration_ns(avg_minarrow_enum),
496 ITERATIONS
497 );
498 println!(
499 "arrow-rs dyn: Int64Array avg = {} (n={})",
500 fmt_duration_ns(avg_arrow_dyn),
501 ITERATIONS
502 );
503
504 println!();
505 println!("|------------ Float Tests (SIMD) --------------|");
506 println!(
507 "raw vec: Vec<f64> avg = {} (n={})",
508 fmt_duration_ns(avg_vec_f64),
509 ITERATIONS
510 );
511 println!(
512 "raw vec64: Vec64<f64> avg = {} (n={})",
513 fmt_duration_ns(avg_vec64_f64),
514 ITERATIONS
515 );
516 println!(
517 "minarrow direct: FloatArray avg = {} (n={})",
518 fmt_duration_ns(avg_minarrow_direct_f64),
519 ITERATIONS
520 );
521 println!(
522 "arrow-rs struct: Float64Array avg = {} (n={})",
523 fmt_duration_ns(avg_arrow_struct_f64),
524 ITERATIONS
525 );
526 println!(
527 "minarrow enum: FloatArray avg = {} (n={})",
528 fmt_duration_ns(avg_minarrow_enum_f64),
529 ITERATIONS
530 );
531 println!(
532 "arrow-rs dyn: Float64Array avg = {} (n={})",
533 fmt_duration_ns(avg_arrow_dyn_f64),
534 ITERATIONS
535 );
536
537 println!("\n=> Vec64 backs the above `Minarrow` types and `Vec` backs Arrow_Rs.");
538
539 println!("\nVerify SIMD pointer alignment for Integer calculations (based on lane width):");
540 println!("Vec<i64> is aligned: {}", v_aligned);
541 println!("Minarrow Vec64<i64> is aligned: {}", v64_aligned);
542 println!(
543 "Minarrow IntegerArray<i64> is aligned: {}",
544 int_array_aligned
545 );
546 println!("Arrow ArrowI64Array is aligned: {}", i64_arrow_aligned);
547 println!(
548 "Minarrow Array::NumericArray<i64> is aligned: {}",
549 arr_int_enum_aligned
550 );
551 println!("Arrow ArrayRef<int> is aligned: {}", array_ref_int_aligned);
552
553 println!("\nVerify SIMD pointer alignment for Float calculations (based on lane width):");
554 println!("Vec<f64> is aligned: {}", v_float_aligned);
555 println!("Vec64<f64> is aligned: {}", v64_float_aligned);
556 println!("FloatArray<f64> is aligned: {}", float_arr_aligned);
557 println!("ArrowF64Array is aligned: {}", arrow_f64_aligned);
558 println!(
559 "Array::NumericArray<f64> is aligned: {}",
560 float_enum_aligned
561 );
562 println!("ArrayRef is aligned: {}", arrow_f64_arr_aligned);
563
564 println!("\n---------------------- END OF SIMD AVG BENCHMARKS ---------------------------");
565 }Sourcepub fn bool(self) -> Result<BooleanArray<u8>, MinarrowError>
pub fn bool(self) -> Result<BooleanArray<u8>, MinarrowError>
Converts to BooleanArray
All non-zero values become true, but the null mask is preserved.
Sourcepub fn str(self) -> Result<StringArray<u32>, MinarrowError>
pub fn str(self) -> Result<StringArray<u32>, MinarrowError>
Converts to StringArray
Preserves Null mask.
Trait Implementations§
Source§impl ByteSize for NumericArray
ByteSize for NumericArray enum
impl ByteSize for NumericArray
ByteSize for NumericArray enum
Source§impl Clone for NumericArray
impl Clone for NumericArray
Source§fn clone(&self) -> NumericArray
fn clone(&self) -> NumericArray
Returns a duplicate of the value. Read more
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreSource§impl Concatenate for NumericArray
impl Concatenate for NumericArray
Source§impl Debug for NumericArray
impl Debug for NumericArray
Source§impl Default for NumericArray
impl Default for NumericArray
Source§fn default() -> NumericArray
fn default() -> NumericArray
Returns the “default value” for a type. Read more
Source§impl Display for NumericArray
impl Display for NumericArray
Source§impl From<NumericArray> for NumericArrayV
impl From<NumericArray> for NumericArrayV
Source§fn from(array: NumericArray) -> Self
fn from(array: NumericArray) -> Self
Converts to this type from the input type.
Source§impl PartialEq for NumericArray
impl PartialEq for NumericArray
Source§impl Shape for NumericArray
impl Shape for NumericArray
impl StructuralPartialEq for NumericArray
Auto Trait Implementations§
impl Freeze for NumericArray
impl RefUnwindSafe for NumericArray
impl Send for NumericArray
impl Sync for NumericArray
impl Unpin for NumericArray
impl UnwindSafe for NumericArray
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> CustomValue for T
impl<T> CustomValue for T
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§impl<T> Key for Twhere
T: Clone,
impl<T> Key for Twhere
T: Clone,
Source§impl<T> Pointable for T
impl<T> Pointable for T
Source§impl<T> PolicyExt for Twhere
T: ?Sized,
impl<T> PolicyExt for Twhere
T: ?Sized,
Source§impl<T> ToCompactString for Twhere
T: Display,
impl<T> ToCompactString for Twhere
T: Display,
Source§fn try_to_compact_string(&self) -> Result<CompactString, ToCompactStringError>
fn try_to_compact_string(&self) -> Result<CompactString, ToCompactStringError>
Fallible version of
ToCompactString::to_compact_string() Read moreSource§fn to_compact_string(&self) -> CompactString
fn to_compact_string(&self) -> CompactString
Converts the given value to a
CompactString. Read moreSource§impl<T> ToStringFallible for Twhere
T: Display,
impl<T> ToStringFallible for Twhere
T: Display,
Source§fn try_to_string(&self) -> Result<String, TryReserveError>
fn try_to_string(&self) -> Result<String, TryReserveError>
ToString::to_string, but without panic on OOM.