Module array 

Zarr arrays.

An array is a node in a Zarr hierarchy used to hold multidimensional array data and associated metadata. See https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html#array.

A Zarr V3 array is defined by the following parameters (which are encoded in its JSON metadata):

• shape: defines the length of the array dimensions,
• data type: defines the numerical representation array elements,
• chunk grid: defines how the array is subdivided into chunks,
• chunk key encoding: defines how chunk grid cell coordinates are mapped to keys in a store,
• fill value: an element value to use for uninitialised portions of the array,
• codecs: used to encode and decode chunks.
• (optional) attributes: user-defined attributes,
• (optional) storage transformers: used to intercept and alter the storage keys and bytes of an array before they reach the underlying physical storage, and
• (optional) dimension names: defines the names of the array dimensions.

See https://zarr-specs.readthedocs.io/en/latest/v3/core/index.html#array-metadata for more information on array metadata.

zarrs supports a subset of Zarr V2 arrays which are a compatible subset of Zarr V3 arrays. This encompasses Zarr V2 array that use supported codecs and could be converted to a Zarr V3 array with only a metadata change.

The documentation for Array details how to interact with arrays.

Re-exports

pub use self::builder::ArrayBuilder;

pub use self::codec::CodecChain;

pub use self::storage_transformer::StorageTransformerChain;

pub use self::storage_transformer::StorageTransformerTraits;

Modules

builder

Array builders.

chunk_cache

Chunk caching.

chunk_grid

Zarr chunk grids.

chunk_key_encoding

Zarr chunk key encodings. Includes a default and v2 implementation.

codec

Zarr codecs.

concurrency

Concurrency utilities for arrays and codecs.

data_type

Zarr data types.

iterators

Array subset iterators.

storage_transformer

Zarr storage transformers.

Structs

AdditionalFieldUnsupportedError

An unsupported additional field error.

Array

A Zarr array.

ArrayBytesFixedDisjointView

A disjoint view of the bytes in an array with a fixed-length data type.

ArrayBytesOffsets

Array element byte offsets.

ArrayBytesOptional

Optional array bytes composed of data and a validity mask.

ArrayBytesRawOffsetsOutOfBoundsError

An error raised if variable length array bytes offsets are out of bounds.

ArrayBytesVariableLength

Variable length array bytes composed of bytes and element bytes offsets.

ArrayMetadataOptions

Options for writing array metadata.

ArrayMetadataV2

Zarr V2 array metadata.

ArrayMetadataV3

Zarr V3 array metadata.

ArrayShardedReadableExtCachesharding

A cache used for methods in the ArrayShardedReadableExt trait.

ArraySubset

An array subset.

AsyncArrayShardedReadableExtCachesharding and async

A cache used for methods in the AsyncArrayShardedReadableExt trait.

ChunkGrid

A chunk grid implementing ChunkGridTraits.

ChunkKeyEncoding

A chunk key encoding.

CodecMetadataOptions

Options for codec metadata.

CodecOptions

Codec options for encoding/decoding.

DataType

A data type implementing DataTypeTraits.

FillValue

A fill value.

IncompatibleDimensionalityError

An incompatible dimensionality error.

RecommendedConcurrency

The recommended concurrency of a codec includes the most efficient and maximum recommended concurrency.

StoragePartialDecoder

A ReadableStorage store value partial decoder.

Tensor

A tensor holding raw bytes with data type and shape metadata.

Enums

ArrayBytes

Fixed or variable length array bytes.

ArrayBytesDecodeIntoTarget

A target for decoding array bytes into a preallocated output view.

ArrayBytesError

Errors related to ArrayBytes<'_> and ArrayBytes.

ArrayBytesFixedDisjointViewCreateError

Errors that can occur when creating a ArrayBytesFixedDisjointView.

ArrayBytesRawOffsetsCreateError

An error creating ArrayBytesOffsets.

ArrayCreateError

An array creation error.

ArrayError

Array errors.

ArrayMetadata

Zarr array metadata (V2 or V3).

ArraySubsetError

An incompatible start/end indices error.

BytesRepresentation

The decoded representation of bytes.

ChunkKeySeparator

A chunk key separator. / or ..

Codec

A generic array to array, array to bytes, or bytes to bytes codec.

CodecError

A codec error.

DataTypeSize

A data type size. Fixed or variable.

ElementError

An element error.

Endianness

Endianness. Either big or little.

FillValueMetadata

Zarr V3 fill value metadata.

IndexerError

An incompatible indexer and array shape error.

TensorError

Errors related to Tensor operations.

Constants

ZARR_NAN_BF16

The Zarr V3 “NaN” fill value for a 16-bit brain floating point number.

ZARR_NAN_F16

The Zarr V3 “NaN” fill value for a 16-bit IEEE 754 floating point number.

ZARR_NAN_F32

The Zarr V3 “NaN” fill value for a 32-bit IEEE 754 floating point number.

ZARR_NAN_F64

The Zarr V3 “NaN” fill value for a 64-bit IEEE 754 floating point number.

Traits

ArrayCodecTraits

Traits for both array to array and array to bytes codecs.

ArrayPartialDecoderTraits

Partial array decoder traits.

ArrayPartialEncoderTraits

Partial array encoder traits.

ArrayShardedExtsharding

An Array extension trait to simplify working with arrays using the sharding_indexed codec.

ArrayShardedReadableExtsharding

An Array extension trait to efficiently read data (e.g. subchunks) from arrays using the sharding_indexed codec.

ArraySubsetTraits

Trait for types that represent an array region (start and shape).

ArrayToArrayCodecTraits

Traits for array to array codecs.

ArrayToBytesCodecTraits

Traits for array to bytes codecs.

AsyncArrayPartialDecoderTraitsasync

Asynchronous partial array decoder traits.

AsyncArrayPartialEncoderTraitsasync

Asynchronous partial array encoder traits.

AsyncArrayShardedReadableExtsharding and async

An Array extension trait to efficiently read data (e.g. subchunks) from arrays using the sharding_indexed codec. s Sharding indexes are cached in a AsyncArrayShardedReadableExtCache enabling faster retrieval.

AsyncBytesPartialDecoderTraitsasync

Asynchronous partial bytes decoder traits.

AsyncBytesPartialEncoderTraitsasync

Asynhronous partial bytes encoder traits.

BytesPartialDecoderTraits

Partial bytes decoder traits.

BytesPartialEncoderTraits

Partial bytes encoder traits.

BytesToBytesCodecTraits

Traits for bytes to bytes codecs.

ChunkGridTraits

Chunk grid traits.

ChunkGridTraitsIterators

Chunk grid iterators.

ChunkKeyEncodingTraits

Chunk key encoding traits.

ChunkShapeTraits

A trait for chunk shapes.

CodecTraits

Codec traits.

CodecTraitsV2

Trait for creating a codec from Zarr V2 metadata.

CodecTraitsV3

Trait for creating a codec from Zarr V3 metadata.

DataTypeTraits

Traits for a data type extension.

DataTypeTraitsV2

Trait for creating a data type from Zarr V2 metadata.

DataTypeTraitsV3

Trait for creating a data type from Zarr V3 metadata.

Element

A trait representing an array element type.

ElementOwned

A trait representing an owned array element type.

FromArrayBytes

A trait for types that can be constructed from ArrayBytes, an array shape and a DataType.

Indexer

A trait for a generic indexer.

IntoArrayBytes

A trait for types that can be converted into ArrayBytes for storage.

Functions

bytes_to_ndarrayndarray

Convert a vector of bytes to an ndarray::ArrayD.

chunk_shape_to_array_shape

Convert a ChunkShape reference to an ArrayShape.

convert_from_bytes_slice

Transmute from &[u8] to Vec<T>.

convert_to_bytes_vec

Convert from &[T] to Vec<u8>.

copy_fill_value_into

Decode the fill value into a subset of a preallocated output.

elements_to_ndarrayndarray

Convert a vector of elements to an ndarray::ArrayD.

ravel_indices

Ravel ND indices to a linearised index.

transmute_from_bytes_vec

Transmute from Vec<u8> to Vec<T>.

transmute_to_bytes

Transmute from &[T] to &[u8].

transmute_to_bytes_vec

Transmute from Vec<T> to Vec<u8>.

unravel_index

Unravel a linearised index to ND indices.

update_array_bytes

Update array bytes.

Type Aliases

ArrayBytesRaw

Raw array element bytes.

ArrayIndices

An ND index to an element in an array or chunk.

ArrayIndicesTinyVec

An ND index to an element in an array or chunk. Uses TinyVec for stack allocation up to 4 dimensions.

ArrayShape

An array shape. Dimensions may be zero.

ChunkShape

A chunk shape. Dimensions must be non-zero.

DimensionName

A dimension name.