1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
//! # `marrow` - minimalist Arrow interop
//!
//! `marrow` allows building and viewing arrow arrays of different implementations using a unified
//! interface. The motivation behind `marrow` is to allow libraries to target multiple different
//! arrow versions simultaneously.
//!
//! Supported arrow implementations:
//!
//! - [`arrow`](https://docs.rs/arrow/)
//! - [`arrow2`](https://docs.rs/arrow2)
//!
//! The main types are
//!
//! - [`Array`][crate::array::Array]: an array with owned data
//! - [`View`][crate::view::View]: an array with borrowed data
//! - [`Field`][crate::datatypes::Field]: the data type and metadata of a field
//! - [`DataType`][crate::datatypes::DataType]: data types of arrays
//!
//! ## Conversions
//!
//! `marrow` offers conversions between its types and the types of different arrow versions. See the
//! [features](#features) section how to enable support for a specific version. The following
//! conversion are implemented.
//!
//! From `marrow` to `arrow`:
//!
//! - `TryFrom<`[`marrow::array::Array`][crate::array::Array]`> for arrow::array::ArrayRef`
//! - `TryFrom<&`[`marrow::datatypes::Field`][crate::datatypes::Field]`> for arrow::datatypes::Field`
//! - `TryFrom<&`[`marrow::datatypes::DataType`][crate::datatypes::DataType]`> for arrow::datatypes::DataType`
//! - `TryFrom<`[`marrow::datatypes::TimeUnit`][crate::datatypes::TimeUnit]`> for arrow::datatypes::TimeUnit`
//! - `TryFrom<`[`marrow::datatypes::UnionMode`][crate::datatypes::UnionMode]`> for arrow::datatypes::UnionMode`
//!
//! From `arrow` to `marrow`:
//!
//! - `TryFrom<&dyn arrow::array::Array> for `[`marrow::view::View<'_>`][crate::view::View]
//! - `TryFrom<&arrow::datatypes::Field> for `[`marrow::datatypes::Field`][crate::datatypes::Field]
//! - `TryFrom<&arrow::datatypes::DataType> for `[`marrow::datatypes::DataType`][crate::datatypes::DataType]
//! - `TryFrom<arrow::datatypes::TimeUnit> for `[`marrow::datatypes::TimeUnit`][crate::datatypes::TimeUnit]
//! - `TryFrom<arrow::datatypes::UnionMode> for `[`marrow::datatypes::UnionMode`][crate::datatypes::UnionMode]
//!
//! For `arrow2` the corresponding conversions are implemented.
//!
//! For example to access the data in an arrow array:
//!
//! ```rust
//! # fn main() -> marrow::error::Result<()> { marrow::_with_arrow! {
//! use arrow::array::Int32Array;
//! use marrow::view::View;
//!
//! // build the arrow array
//! let arrow_array = Int32Array::from(vec![Some(1), Some(2), Some(3)]);
//!
//! // construct a view of this array
//! let marrow_view = View::try_from(&arrow_array as &dyn arrow::array::Array)?;
//!
//! // access the underlying data
//! let View::Int32(marrow_view) = marrow_view else { panic!() };
//! assert_eq!(marrow_view.values, &[1, 2, 3]);
//! # } }
//! ```
//!
//! Or to build an array:
//!
//! ```rust
//! # fn main() -> marrow::error::Result<()> { marrow::_with_arrow! {
//! use arrow::array::Array as _;
//! use marrow::array::{Array, PrimitiveArray};
//!
//! // build the array
//! let marrow_array = Array::Int32(PrimitiveArray {
//! validity: Some(marrow::bit_vec![true, false, true]),
//! values: vec![4, 0, 6],
//! });
//!
//! // convert it to an arrow array
//! let arrow_array_ref = arrow::array::ArrayRef::try_from(marrow_array)?;
//! assert_eq!(arrow_array_ref.is_null(0), false);
//! assert_eq!(arrow_array_ref.is_null(1), true);
//! assert_eq!(arrow_array_ref.is_null(2), false);
//! # } }
//! ```
//!
//! ## Features
//!
//! Supported features:
//!
//! - `serde`: enable Serde serialization / deserialization for schema types
//! ([Field][crate::datatypes::Field], [DataType][crate::datatypes::DataType], ...). The format
//! will match the `arrow` crate
//! - `arrow-{version}`: enable conversions between `marrow` and `arrow={version}`
//! - `arrow2-{version}`: enable conversions between `marrow` and `arrow2={version}`
//!
//! This crate supports conversions from and to different version of `arrow` or `arrow2`. These
//! conversions can be enabled by selecting the relevant features. Any combination of features can
//! be selected, e.g., both `arrow-53` and `arrow-52` can be used at the same time.
//!
//! Supported arrow versions:
//!
//! | Feature | Arrow Version |
//! |---------------|---------------|
// arrow-version:insert: //! | `arrow-{version}` | `arrow={version}` |
//! | `arrow-58` | `arrow=58` |
//! | `arrow-56` | `arrow=56` |
//! | `arrow-55` | `arrow=55` |
//! | `arrow-54` | `arrow=54` |
//! | `arrow-53` | `arrow=53` |
//! | `arrow-52` | `arrow=52` |
//! | `arrow-51` | `arrow=51` |
//! | `arrow-50` | `arrow=50` |
//! | `arrow-49` | `arrow=49` |
//! | `arrow-48` | `arrow=48` |
//! | `arrow-47` | `arrow=47` |
//! | `arrow-46` | `arrow=46` |
//! | `arrow-45` | `arrow=45` |
//! | `arrow-44` | `arrow=44` |
//! | `arrow-43` | `arrow=43` |
//! | `arrow-42` | `arrow=42` |
//! | `arrow-41` | `arrow=41` |
//! | `arrow-40` | `arrow=40` |
//! | `arrow-39` | `arrow=39` |
//! | `arrow-38` | `arrow=38` |
//! | `arrow-37` | `arrow=37` |
//! | `arrow2-0-17` | `arrow2=0.17` |
//! | `arrow2-0-16` | `arrow2=0.16` |
//!
//! Note, `arrow2=0.18` is not supported as the source code was not tagged on GitHub.
//!