Crate nd_arrow_array 

N-dimensional arrays as an Arrow extension type.

• NdArrowArray: one N-dimensional array: flat values, named Dimensions and Attributes.
• NdArrayType: the nd_arrow.array Arrow extension type. Its storage is Struct { values, dim_names, dim_sizes, attrs }.
• NdColumn: an Arrow column of the extension type. Each row is one NdArrowArray.
• NdRecordBatch: nd arrays over a shared target grid, encoded as a RecordBatch of nd columns.
• compute: kernels with NumPy and xarray semantics (layout, arithmetic, reductions, search, interpolation, CF decoding).
• tensor: conversion to and from the Arrow canonical tensor types.

Features: parallel runs large gathers and reductions with rayon; ndarray adds conversion to and from ndarray::ArrayD.

Example

use std::sync::Arc;

use arrow::array::Float64Array;
use nd_arrow_array::compute::{Selector, Slice};
use nd_arrow_array::{Attributes, Dimensions, NdArrowArray, NdColumn};

let sst = NdArrowArray::try_new(
    Arc::new(Float64Array::from(vec![10.0, 11.0, 12.0, 14.0, 15.0, 16.0])),
    Dimensions::try_new([("time", 2), ("lat", 3)])?,
)?
.with_attributes(Attributes::new().with("units", "degC"));

// Anomaly against the time mean: the mean broadcasts back by name.
let anomaly = (&sst - &sst.mean(Some(&["time"]))?)?;
let north = anomaly.isel(&[("lat", Selector::Slice(Slice::range(1, 3)))])?;
assert_eq!(north.shape(), vec![2, 2]);
println!("{north}");

// One row of an Arrow column with the `nd_arrow.array` extension type.
let column = NdColumn::try_from_arrays([&sst])?;
assert_eq!(column.value(0), sst);

Re-exports

pub use array::NdArrowArray;

pub use attributes::Attributes;

pub use attributes::IntoScalar;

pub use batch::NdRecordBatch;

pub use batch::encode_batches;

pub use batch::infer_target;

pub use column::NdColumn;

pub use column::NdColumnBuilder;

pub use dimensions::Dimension;

pub use dimensions::Dimensions;

pub use error::NdError;

pub use error::Result;

pub use extension::NdArrayMetadata;

pub use extension::NdArrayType;

Modules

array

A single N-dimensional Arrow array.

attributes

Per-array attributes (CF/netCDF style metadata such as units).

batch

Grid-shaped record batches: columns on subsets of one shared target grid.

column

An Arrow column of nd arrays: the storage array of NdArrayType.

compute

Compute kernels for NdArrowArray, with NumPy and xarray semantics.

dimensions

Named dimensions for N-dimensional Arrow arrays.

error

Error type for nd arrays.

extension

The nd_arrow.array Arrow extension type.

tensor

Conversion between nd columns and the Arrow canonical tensor types arrow.fixed_shape_tensor and arrow.variable_shape_tensor.