Struct Array 

pub struct Array<TStorage: ?Sized> { /* private fields */ }

A Zarr array.

Initilisation

The easiest way to create a new Zarr V3 array is with an ArrayBuilder. Alternatively, a new Zarr V2 or Zarr V3 array can be created with Array::new_with_metadata.

An existing Zarr V2 or Zarr V3 array can be initialised with Array::open or Array::open_opt with metadata read from the store.

Array initialisation will error if ArrayMetadata contains:

• unsupported extension points, including extensions which are supported by zarrs but have not been enabled with the appropriate features gates, or
• incompatible codecs (e.g. codecs in wrong order, codecs incompatible with data type, etc.),
• a chunk grid incompatible with the array shape,
• a fill value incompatible with the data type, or
• the metadata is in invalid in some other way.

Array Metadata

Array metadata must be explicitly stored with store_metadata or store_metadata_opt if an array is newly created or its metadata has been mutated.

The underlying metadata of an Array can be accessed with metadata or metadata_opt. The latter accepts ArrayMetadataOptions that can be used to convert array metadata from Zarr V2 to V3, for example. metadata_opt is used internally by store_metadata / store_metadata_opt. Use serde_json::to_string or serde_json::to_string_pretty on ArrayMetadata to convert it to a JSON string.

Immutable Array Metadata / Properties

• metadata: the underlying ArrayMetadata structure containing all array metadata
• data_type
• fill_value
• chunk_grid
• chunk_key_encoding
• codecs
• storage_transformers
• path

Mutable Array Metadata

Do not forget to store metadata after mutation.

• shape / set_shape / set_shape_and_chunk_grid
• attributes / attributes_mut
• dimension_names / set_dimension_names

zarrs Metadata

By default, the zarrs version and a link to its source code is written to the _zarrs attribute in array metadata when calling store_metadata. Override this behaviour globally with Config::set_include_zarrs_metadata or call store_metadata_opt with an explicit ArrayMetadataOptions.

Array Data

Array operations are divided into several categories based on the traits implemented for the backing storage. The core array methods are:

• [Async]ReadableStorageTraits: read array data and metadata
  • retrieve_chunk_if_exists
  • retrieve_chunk
  • retrieve_chunks
  • retrieve_chunk_subset
  • retrieve_array_subset
  • retrieve_encoded_chunk
  • partial_decoder
• [Async]WritableStorageTraits: store/erase array data and metadata
  • store_metadata
  • erase_metadata
  • store_chunk
  • store_chunks
  • store_encoded_chunk
  • erase_chunk
  • erase_chunks
• [Async]ReadableWritableStorageTraits: store operations requiring reading and writing
  • store_chunk_subset
  • store_array_subset
  • partial_encoder

Many retrieve and store methods have a standard and an _opt variant. The latter has an additional CodecOptions parameter for fine-grained concurrency control and more.

Array retrieve_* methods are generic over the return type. For example, the following variants are available for retrieving chunks or array subsets:

• Raw bytes variants: ArrayBytes
• Typed element variants: e.g. Vec<T> where T: Element
• ndarray variants: ndarray::ArrayD<T> where T: Element (requires ndarray feature)
• dlpack variants: RawBytesDlPack where T: Element (requires dlpack feature)

Similarly, array store_* methods are generic over the input type.

async_ prefix variants can be used with async stores (requires async feature).

Additional Array methods are offered by extension traits:

• ArrayShardedExt and ArrayShardedReadableExt: see Reading Sharded Arrays.

ChunkCache implementations offer a similar API to Array::ReadableStorageTraits, except with Chunk Caching support.

Chunks and Array Subsets

Several convenience methods are available for querying the underlying chunk grid:

• chunk_origin
• chunk_shape
• chunk_subset
• chunk_subset_bounded
• chunks_subset / chunks_subset_bounded
• chunks_in_array_subset

An ArraySubset spanning the entire array can be retrieved with subset_all.

Example: Update an Array Chunk-by-Chunk (in Parallel)

In the below example, an array is updated chunk-by-chunk in parallel. This makes use of chunk_subset_bounded to retrieve and store only the subset of chunks that are within the array bounds. This can occur when a regular chunk grid does not evenly divide the array shape, for example.

// Get an iterator over the chunk indices
//   The array shape must have been set (i.e. non-zero), otherwise the
//   iterator will be empty
let chunk_grid_shape = array.chunk_grid_shape();
let chunks: Indices = ArraySubset::new_with_shape(chunk_grid_shape.to_vec()).indices();

// Iterate over chunk indices (in parallel)
chunks.into_par_iter().try_for_each(|chunk_indices: ArrayIndicesTinyVec| {
    // Retrieve the array subset of the chunk within the array bounds
    //   This partially decodes chunks that extend beyond the array end
    let subset: ArraySubset = array.chunk_subset_bounded(&chunk_indices)?;
    let chunk_bytes: ArrayBytes = array.retrieve_array_subset(&subset)?;

    // ... Update the chunk bytes

    // Write the updated chunk
    //   Elements beyond the array bounds in straddling chunks are left
    //   unmodified or set to the fill value if the chunk did not exist.
    array.store_array_subset(&subset, chunk_bytes)
})?;

Optimising Writes

For optimum write performance, an array should be written using store_chunk or store_chunks where possible.

store_chunk_subset and store_array_subset may incur decoding overhead, and they require careful usage if executed in parallel (see Parallel Writing below). However, these methods will use a fast path and avoid decoding if the subset covers entire chunks.

Direct IO (Linux)

If using Linux, enabling direct IO with the FilesystemStore may improve write performance.

Currently, the most performant path for uncompressed writing is to reuse page aligned buffers via store_encoded_chunk. See zarrs GitHub issue #58 for a discussion on this method.

Parallel Writing

zarrs does not currently offer a “synchronisation” API for locking chunks or array subsets.

It is the responsibility of zarrs consumers to ensure that chunks are not written to concurrently.

If a chunk is written more than once, its element values depend on whichever operation wrote to the chunk last. The store_chunk_subset and store_array_subset methods and their variants internally retrieve, update, and store chunks. So do partial_encoders, which may used internally by the above methods.

It is the responsibility of zarrs consumers to ensure that:

• store_array_subset is not called concurrently on array subsets sharing chunks,
• store_chunk_subset is not called concurrently on the same chunk,
• partial_encoders are created or used concurrently for the same chunk,
• or any combination of the above are called concurrently on the same chunk.

Partial writes to a chunk may be lost if these rules are not respected.

Optimising Reads

It is fastest to load arrays using retrieve_chunk or retrieve_chunks where possible. In contrast, the retrieve_chunk_subset and retrieve_array_subset may use partial decoders which can be less efficient with some codecs/stores. Like their write counterparts, these methods will use a fast path if subsets cover entire chunks.

Standard Array retrieve methods do not perform any caching. For this reason, retrieving multiple subsets in a chunk with retrieve_chunk_subset is very inefficient and strongly discouraged. For example, consider that a compressed chunk may need to be retrieved and decoded in its entirety even if only a small part of the data is needed. In such situations, prefer to initialise a partial decoder for a chunk with partial_decoder and then retrieve multiple chunk subsets with partial_decode. The underlying codec chain will use a cache where efficient to optimise multiple partial decoding requests (see CodecChain). Another alternative is to use Chunk Caching.

Chunk Caching

Refer to the chunk_cache module.

Reading Sharded Arrays

The sharding_indexed codec (ShardingCodec) enables multiple subchunks (“inner chunks”) to be stored in a single chunk (“shard”). With a sharded array, the chunk_grid and chunk indices in store/retrieve methods reference the chunks (“shards”) of an array.

The ArrayShardedExt trait provides additional methods to Array to query if an array is sharded and retrieve the subchunk shape. Additionally, the subchunk grid can be queried, which is a ChunkGrid where chunk indices refer to subchunks rather than shards.

The ArrayShardedReadableExt trait adds Array methods to conveniently and efficiently access the data in a sharded array (with _elements and _ndarray variants):

• retrieve_subchunk_opt
• retrieve_subchunks_opt
• retrieve_array_subset_sharded_opt

For unsharded arrays, these methods gracefully fallback to referencing standard chunks. Each method has a cache parameter (ArrayShardedReadableExtCache) that stores shard indexes so that they do not have to be repeatedly retrieved and decoded.

Parallelism and Concurrency

Sync API

Codecs run in parallel using a dedicated threadpool. Array store and retrieve methods will also run in parallel when they involve multiple chunks. zarrs will automatically choose where to prioritise parallelism between codecs/chunks based on the codecs and number of chunks.

By default, all available CPU cores will be used (where possible/efficient). Concurrency can be limited globally with Config::set_codec_concurrent_target or as required using _opt methods with CodecOptions manipulated with CodecOptions::set_concurrent_target.

Async API

This crate is async runtime-agnostic. Async methods do not spawn tasks internally, so asynchronous storage calls are concurrent but not parallel. Codec encoding and decoding operations still execute in parallel (where supported) in an asynchronous context.

Due the lack of parallelism, methods like async_retrieve_array_subset or async_retrieve_chunks do not parallelise over chunks and can be slow compared to the sync API. Parallelism over chunks can be achieved by spawning tasks outside of zarrs. A crate like async-scoped can enable spawning non-'static futures. If executing many tasks concurrently, consider reducing the codec concurrent_target.

Custom Extensions

zarrs can be extended with custom data types, codecs, chunk grids, chunk key encodings, and storage transformers. The best way to learn how to create extensions is to study the source code of existing extensions in the zarrs crate. The zarrs book also has a chapter on creating custom extensions.

zarrs uses a plugin system to create extension point implementations (e.g. data types, codecs, chunk grids, chunk key encodings, and storage transformers) from metadata. Plugins are registered at compile time using the inventory crate. Runtime plugins are also supported, which take precedence over compile-time plugins. Each plugin has a name matching function that identifies whether it should handle given metadata.

Extensions support name aliases, which can be tied to specific Zarr versions. This allows experimental codecs (e.g. zarrs.zfp) to be later promoted to registered Zarr codecs (e.g. zfp) without breaking support for older arrays. The aliasing system allows matching against string aliases or regex patterns.

The key traits for each extension type are:

• Data types (zarrs_data_type): DataTypeTraits, DataTypeTraitsV2, DataTypeTraitsV3
• Codecs (zarrs_codec): CodecTraits, CodecTraitsV2, CodecTraitsV3)
  • Array-to-array codecs: ArrayCodecTraits + ArrayToArrayCodecTraits
    • ArrayPartialEncoderTraits, AsyncArrayPartialEncoderTraits
    • ArrayPartialDecoderTraits, AsyncArrayPartialDecoderTraits
  • Array-to-bytes codecs: ArrayCodecTraits + ArrayToBytesCodecTraits
    • BytesPartialEncoderTraits, AsyncBytesPartialEncoderTraits
    • BytesPartialDecoderTraits, AsyncBytesPartialDecoderTraits
  • Bytes-to-bytes codecs: BytesToBytesCodecTraits
    • BytesPartialEncoderTraits, AsyncBytesPartialEncoderTraits
    • BytesPartialDecoderTraits, AsyncBytesPartialDecoderTraits
  • Chunk grids (zarrs_chunk_grid): ChunkGridTraits
  • Chunk key encodings (zarrs_chunk_key_encoding): ChunkKeyEncodingTraits
  • Storage transformers (crate::array::storage_transformer): StorageTransformerTraits

Extensions are registered via the following:

• Data types: DataTypePluginV2, DataTypePluginV3 DataTypeRuntimePluginV2, DataTypeRuntimePluginV3
• Codecs: CodecPluginV3, CodecPluginV2, CodecRuntimePluginV2, CodecRuntimePluginV3
• Chunk grids: ChunkGridPlugin, ChunkGridRuntimePlugin
• Chunk key encodings: ChunkKeyEncodingPlugin, ChunkKeyEncodingRuntimePlugin
• Storage transformers: StorageTransformerPlugin, StorageTransformerRuntimePlugin

Implementations

impl<TStorage: ?Sized + ReadableStorageTraits + 'static> Array<TStorage>

pub fn open( storage: Arc<TStorage>, path: &str, ) -> Result<Self, ArrayCreateError>

Open an existing array in storage at path with default MetadataRetrieveVersion. The metadata is read from the store.

Errors

Returns ArrayCreateError if there is a storage error or any metadata is invalid.

Examples found in repository

examples/sync_http_array_read.rs (line 65)

fn http_array_read(backend: Backend) -> Result<(), Box<dyn std::error::Error>> {
    const HTTP_URL: &str =
        "https://raw.githubusercontent.com/zarrs/zarrs/main/zarrs/tests/data/array_write_read.zarr";
    const ARRAY_PATH: &str = "/group/array";

    // Create a HTTP store
    // let mut store: ReadableStorage = Arc::new(store::HTTPStore::new(HTTP_URL)?);
    let block_on = TokioBlockOn(tokio::runtime::Runtime::new()?);
    let mut store: ReadableStorage = match backend {
        // Backend::OpenDAL => {
        //     let builder = opendal::services::Http::default().endpoint(HTTP_URL);
        //     let operator = opendal::Operator::new(builder)?.finish();
        //     let store = Arc::new(zarrs_opendal::AsyncOpendalStore::new(operator));
        //     Arc::new(AsyncToSyncStorageAdapter::new(store, block_on))
        // }
        Backend::ObjectStore => {
            let options = object_store::ClientOptions::new().with_allow_http(true);
            let store = object_store::http::HttpBuilder::new()
                .with_url(HTTP_URL)
                .with_client_options(options)
                .build()?;
            let store = Arc::new(zarrs_object_store::AsyncObjectStore::new(store));
            Arc::new(AsyncToSyncStorageAdapter::new(store, block_on))
        }
    };
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Init the existing array, reading metadata
    let array = Array::open(store, ARRAY_PATH)?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Read the whole array
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&array.subset_all())?;
    println!("The whole array is:\n{data_all}\n");

    // Read a chunk back from the store
    let chunk_indices = vec![1, 0];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("Chunk [1,0] is:\n{data_chunk}\n");

    // Read the central 4x2 subset of the array
    let subset_4x2 = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_4x2: ArrayD<f32> = array.retrieve_array_subset(&subset_4x2)?;
    println!("The middle 4x2 subset is:\n{data_4x2}\n");

    Ok(())
}

pub fn open_opt( storage: Arc<TStorage>, path: &str, version: &MetadataRetrieveVersion, ) -> Result<Self, ArrayCreateError>

Open an existing array in storage at path with non-default MetadataRetrieveVersion. The metadata is read from the store.

Errors

Returns ArrayCreateError if there is a storage error or any metadata is invalid.

pub fn retrieve_chunk_if_exists<T: FromArrayBytes>( &self, chunk_indices: &[u64], ) -> Result<Option<T>, ArrayError>

Read and decode the chunk at chunk_indices into its bytes if it exists with default codec options.

Errors

Returns an ArrayError if

• chunk_indices are invalid,
• there is a codec decoding error, or
• an underlying store error.

Panics

Panics if the number of elements in the chunk exceeds usize::MAX.

pub fn retrieve_chunk_elements_if_exists<T: ElementOwned>( &self, chunk_indices: &[u64], ) -> Result<Option<Vec<T>>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_chunk_if_exists::<Vec<T>>() instead

Read and decode the chunk at chunk_indices into a vector of its elements if it exists with default codec options.

Errors

Returns an ArrayError if

• the size of T does not match the data type size,
• the decoded bytes cannot be transmuted,
• chunk_indices are invalid,
• there is a codec decoding error, or
• an underlying store error.

pub fn retrieve_chunk_ndarray_if_exists<T: ElementOwned>( &self, chunk_indices: &[u64], ) -> Result<Option<ArrayD<T>>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_chunk_if_exists::<ndarray::ArrayD<T>>() instead

Available on crate feature ndarray only.

Read and decode the chunk at chunk_indices into an ndarray::ArrayD if it exists.

Errors

Returns an ArrayError if:

• the size of T does not match the data type size,
• the decoded bytes cannot be transmuted,
• the chunk indices are invalid,
• there is a codec decoding error, or
• an underlying store error.

Panics

Will panic if a chunk dimension is larger than usize::MAX.

pub fn retrieve_encoded_chunk( &self, chunk_indices: &[u64], ) -> Result<Option<Vec<u8>>, StorageError>

Retrieve the encoded bytes of a chunk.

Errors

Returns an StorageError if there is an underlying store error.

pub fn retrieve_chunk<T: FromArrayBytes>( &self, chunk_indices: &[u64], ) -> Result<T, ArrayError>

Read and decode the chunk at chunk_indices into its bytes or the fill value if it does not exist with default codec options.

Errors

Returns an ArrayError if

• chunk_indices are invalid,
• there is a codec decoding error, or
• an underlying store error.

Panics

Panics if the number of elements in the chunk exceeds usize::MAX.

Examples found in repository

examples/sync_http_array_read.rs (line 78)

fn http_array_read(backend: Backend) -> Result<(), Box<dyn std::error::Error>> {
    const HTTP_URL: &str =
        "https://raw.githubusercontent.com/zarrs/zarrs/main/zarrs/tests/data/array_write_read.zarr";
    const ARRAY_PATH: &str = "/group/array";

    // Create a HTTP store
    // let mut store: ReadableStorage = Arc::new(store::HTTPStore::new(HTTP_URL)?);
    let block_on = TokioBlockOn(tokio::runtime::Runtime::new()?);
    let mut store: ReadableStorage = match backend {
        // Backend::OpenDAL => {
        //     let builder = opendal::services::Http::default().endpoint(HTTP_URL);
        //     let operator = opendal::Operator::new(builder)?.finish();
        //     let store = Arc::new(zarrs_opendal::AsyncOpendalStore::new(operator));
        //     Arc::new(AsyncToSyncStorageAdapter::new(store, block_on))
        // }
        Backend::ObjectStore => {
            let options = object_store::ClientOptions::new().with_allow_http(true);
            let store = object_store::http::HttpBuilder::new()
                .with_url(HTTP_URL)
                .with_client_options(options)
                .build()?;
            let store = Arc::new(zarrs_object_store::AsyncObjectStore::new(store));
            Arc::new(AsyncToSyncStorageAdapter::new(store, block_on))
        }
    };
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Init the existing array, reading metadata
    let array = Array::open(store, ARRAY_PATH)?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Read the whole array
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&array.subset_all())?;
    println!("The whole array is:\n{data_all}\n");

    // Read a chunk back from the store
    let chunk_indices = vec![1, 0];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("Chunk [1,0] is:\n{data_chunk}\n");

    // Read the central 4x2 subset of the array
    let subset_4x2 = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_4x2: ArrayD<f32> = array.retrieve_array_subset(&subset_4x2)?;
    println!("The middle 4x2 subset is:\n{data_4x2}\n");

    Ok(())
}

examples/rectangular_array_write_read.rs (line 140)

fn rectangular_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use rayon::prelude::{IntoParallelIterator, ParallelIterator};
    use zarrs::array::{ArraySubset, ZARR_NAN_F32, codec, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        MetadataV3::new_with_configuration(
            "rectangular",
            RectangularChunkGridConfiguration {
                chunk_shape: vec![
                    vec![
                        NonZeroU64::new(1).unwrap(),
                        NonZeroU64::new(2).unwrap(),
                        NonZeroU64::new(3).unwrap(),
                        NonZeroU64::new(2).unwrap(),
                    ]
                    .into(),
                    NonZeroU64::new(4).unwrap().into(),
                ], // chunk sizes
            },
        ),
        data_type::float32(),
        ZARR_NAN_F32,
    )
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(codec::GzipCodec::new(5)?),
    ])
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    // Write some chunks (in parallel)
    (0..4).into_par_iter().try_for_each(|i| {
        let chunk_grid = array.chunk_grid();
        let chunk_indices = vec![i, 0];
        if let Some(chunk_shape) = chunk_grid.chunk_shape(&chunk_indices)? {
            let chunk_array = ndarray::ArrayD::<f32>::from_elem(
                chunk_shape
                    .iter()
                    .map(|u| u.get() as usize)
                    .collect::<Vec<_>>(),
                i as f32,
            );
            array.store_chunk(&chunk_indices, chunk_array)
        } else {
            Err(zarrs::array::ArrayError::InvalidChunkGridIndicesError(
                chunk_indices.to_vec(),
            ))
        }
    })?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write a subset spanning multiple chunks, including updating chunks already written
    array.store_array_subset(
        &[3..6, 3..6], // start
        ndarray::ArrayD::<f32>::from_shape_vec(
            vec![3, 3],
            vec![0.1f32, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9],
        )?,
    )?;

    // Store elements directly, in this case set the 7th column to 123.0
    array.store_array_subset(&[0..8, 6..7], &[123.0f32; 8])?;

    // Store elements directly in a chunk, in this case set the last row of the bottom right chunk
    array.store_chunk_subset(
        // chunk indices
        &[3, 1],
        // subset within chunk
        &[1..2, 0..4],
        &[-4.0f32; 4],
    )?;

    // Read the whole array
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&array.subset_all())?;
    println!("The whole array is:\n{data_all}\n");

    // Read a chunk back from the store
    let chunk_indices = vec![1, 0];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("Chunk [1,0] is:\n{data_chunk}\n");

    // Read the central 4x2 subset of the array
    let subset_4x2 = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_4x2: ArrayD<f32> = array.retrieve_array_subset(&subset_4x2)?;
    println!("The middle 4x2 subset is:\n{data_4x2}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("The Zarr hierarchy tree is:\n{tree}");

    Ok(())
}

examples/array_write_read.rs (line 138)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    (0..2).into_par_iter().try_for_each(|i| {
        let chunk_indices: Vec<u64> = vec![0, i];
        let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
            zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
        })?;
        array.store_chunk(
            &chunk_indices,
            vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
        )
    })?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array.store_chunks(
        &[1..2, 0..2],
        &[
            //
            1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            //
            1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
        ],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array.store_array_subset(
        &[3..6, 3..6],
        &[-3.3f32, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array.store_array_subset(
        &[0..8, 6..7],
        &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array.store_chunk_subset(
        // chunk indices
        &[1, 1],
        // subset within chunk
        &[3..4, 0..4],
        &[-7.4f32, -7.5, -7.6, -7.7],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.erase_chunk(&[0, 0])?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.retrieve_chunks(&chunks)?;
    println!("retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.retrieve_array_subset(&subset)?;
    println!("retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

examples/sharded_array_write_read.rs (line 112)

fn sharded_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use rayon::prelude::{IntoParallelIterator, ParallelIterator};
    use zarrs::array::{ArraySubset, codec, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/sharded_array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    // Create an array
    let array_path = "/group/array";
    let subchunk_shape = vec![4, 4];
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 8], // chunk (shard) shape
        data_type::uint16(),
        0u16,
    )
    .subchunk_shape(subchunk_shape.clone())
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(codec::GzipCodec::new(5)?),
    ])
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    // The array metadata is
    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Use default codec options (concurrency etc)
    let options = CodecOptions::default();

    // Write some shards (in parallel)
    (0..2).into_par_iter().try_for_each(|s| {
        let chunk_grid = array.chunk_grid();
        let chunk_indices = vec![s, 0];
        if let Some(chunk_shape) = chunk_grid.chunk_shape(&chunk_indices)? {
            let chunk_array = ndarray::ArrayD::<u16>::from_shape_fn(
                chunk_shape
                    .iter()
                    .map(|u| u.get() as usize)
                    .collect::<Vec<_>>(),
                |ij| {
                    (s * chunk_shape[0].get() * chunk_shape[1].get()
                        + ij[0] as u64 * chunk_shape[1].get()
                        + ij[1] as u64) as u16
                },
            );
            array.store_chunk(&chunk_indices, chunk_array)
        } else {
            Err(zarrs::array::ArrayError::InvalidChunkGridIndicesError(
                chunk_indices.to_vec(),
            ))
        }
    })?;

    // Read the whole array
    let data_all: ArrayD<u16> = array.retrieve_array_subset(&array.subset_all())?;
    println!("The whole array is:\n{data_all}\n");

    // Read a shard back from the store
    let shard_indices = vec![1, 0];
    let data_shard: ArrayD<u16> = array.retrieve_chunk(&shard_indices)?;
    println!("Shard [1,0] is:\n{data_shard}\n");

    // Read a subchunk from the store
    let subset_chunk_1_0 = ArraySubset::new_with_ranges(&[4..8, 0..4]);
    let data_chunk: ArrayD<u16> = array.retrieve_array_subset(&subset_chunk_1_0)?;
    println!("Chunk [1,0] is:\n{data_chunk}\n");

    // Read the central 4x2 subset of the array
    let subset_4x2 = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_4x2: ArrayD<u16> = array.retrieve_array_subset(&subset_4x2)?;
    println!("The middle 4x2 subset is:\n{data_4x2}\n");

    // Decode subchunks
    // In some cases, it might be preferable to decode subchunks in a shard directly.
    // If using the partial decoder, then the shard index will only be read once from the store.
    let partial_decoder = array.partial_decoder(&[0, 0])?;
    println!("Decoded subchunks:");
    for subchunk_subset in [
        ArraySubset::new_with_start_shape(vec![0, 0], subchunk_shape.clone())?,
        ArraySubset::new_with_start_shape(vec![0, 4], subchunk_shape.clone())?,
    ] {
        println!("{subchunk_subset}");
        let decoded_subchunk_bytes = partial_decoder.partial_decode(&subchunk_subset, &options)?;
        let ndarray = bytes_to_ndarray::<u16>(
            &subchunk_shape,
            decoded_subchunk_bytes.into_fixed()?.into_owned(),
        )?;
        println!("{ndarray}\n");
    }

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("The Zarr hierarchy tree is:\n{}", tree);

    println!(
        "The keys in the store are:\n[{}]",
        store.list().unwrap_or_default().iter().format(", ")
    );

    Ok(())
}

examples/array_write_read_ndarray.rs (line 147)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    (0..2).into_par_iter().try_for_each(|i| {
        let chunk_indices: Vec<u64> = vec![0, i];
        let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
            zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
        })?;
        array.store_chunk(
            &chunk_indices,
            ArrayD::<f32>::from_shape_vec(
                chunk_subset.shape_usize(),
                vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
            )
            .unwrap(),
        )
    })?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    let ndarray_chunks: Array2<f32> = array![
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
    ];
    array.store_chunks(&[1..2, 0..2], ndarray_chunks)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    let ndarray_subset: Array2<f32> =
        array![[-3.3, -3.4, -3.5,], [-4.3, -4.4, -4.5,], [-5.3, -5.4, -5.5],];
    array.store_array_subset(&[3..6, 3..6], ndarray_subset)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    let ndarray_subset: Array2<f32> = array![
        [-0.6],
        [-1.6],
        [-2.6],
        [-3.6],
        [-4.6],
        [-5.6],
        [-6.6],
        [-7.6],
    ];
    array.store_array_subset(&[0..8, 6..7], ndarray_subset)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    let ndarray_chunk_subset: Array2<f32> = array![[-7.4, -7.5, -7.6, -7.7],];
    array.store_chunk_subset(
        // chunk indices
        &[1, 1],
        // subset within chunk
        &[3..4, 0..4],
        ndarray_chunk_subset,
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.erase_chunk(&[0, 0])?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.retrieve_chunks(&chunks)?;
    println!("retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.retrieve_array_subset(&subset)?;
    println!("retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

pub fn retrieve_chunk_elements<T: ElementOwned>( &self, chunk_indices: &[u64], ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_chunk::<Vec<T>>() instead

Read and decode the chunk at chunk_indices into a vector of its elements or the fill value if it does not exist.

Errors

Returns an ArrayError if

• the size of T does not match the data type size,
• the decoded bytes cannot be transmuted,
• chunk_indices are invalid,
• there is a codec decoding error, or
• an underlying store error.

pub fn retrieve_chunk_ndarray<T: ElementOwned>( &self, chunk_indices: &[u64], ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_chunk::<ndarray::ArrayD<T>>() instead

Available on crate feature ndarray only.

Read and decode the chunk at chunk_indices into an ndarray::ArrayD. It is filled with the fill value if it does not exist.

Errors

Returns an ArrayError if:

• the size of T does not match the data type size,
• the decoded bytes cannot be transmuted,
• the chunk indices are invalid,
• there is a codec decoding error, or
• an underlying store error.

Panics

Will panic if a chunk dimension is larger than usize::MAX.

pub fn retrieve_encoded_chunks( &self, chunks: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<Vec<Option<Vec<u8>>>, StorageError>

Retrieve the encoded bytes of the chunks in chunks.

The chunks are in order of the chunk indices returned by chunks.indices().into_iter().

Errors

Returns a StorageError if there is an underlying store error.

pub fn retrieve_chunks<T: FromArrayBytes>( &self, chunks: &dyn ArraySubsetTraits, ) -> Result<T, ArrayError>

Read and decode the chunks at chunks into their bytes.

Errors

Returns an ArrayError if

• any chunk indices in chunks are invalid,
• there is a codec decoding error, or
• an underlying store error.

Panics

Panics if the number of array elements in the chunk exceeds usize::MAX.

Examples found in repository

examples/array_write_read.rs (line 143)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    (0..2).into_par_iter().try_for_each(|i| {
        let chunk_indices: Vec<u64> = vec![0, i];
        let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
            zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
        })?;
        array.store_chunk(
            &chunk_indices,
            vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
        )
    })?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array.store_chunks(
        &[1..2, 0..2],
        &[
            //
            1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            //
            1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
        ],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array.store_array_subset(
        &[3..6, 3..6],
        &[-3.3f32, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array.store_array_subset(
        &[0..8, 6..7],
        &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array.store_chunk_subset(
        // chunk indices
        &[1, 1],
        // subset within chunk
        &[3..4, 0..4],
        &[-7.4f32, -7.5, -7.6, -7.7],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.erase_chunk(&[0, 0])?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.retrieve_chunks(&chunks)?;
    println!("retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.retrieve_array_subset(&subset)?;
    println!("retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

examples/array_write_read_ndarray.rs (line 152)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    (0..2).into_par_iter().try_for_each(|i| {
        let chunk_indices: Vec<u64> = vec![0, i];
        let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
            zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
        })?;
        array.store_chunk(
            &chunk_indices,
            ArrayD::<f32>::from_shape_vec(
                chunk_subset.shape_usize(),
                vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
            )
            .unwrap(),
        )
    })?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    let ndarray_chunks: Array2<f32> = array![
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
    ];
    array.store_chunks(&[1..2, 0..2], ndarray_chunks)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    let ndarray_subset: Array2<f32> =
        array![[-3.3, -3.4, -3.5,], [-4.3, -4.4, -4.5,], [-5.3, -5.4, -5.5],];
    array.store_array_subset(&[3..6, 3..6], ndarray_subset)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    let ndarray_subset: Array2<f32> = array![
        [-0.6],
        [-1.6],
        [-2.6],
        [-3.6],
        [-4.6],
        [-5.6],
        [-6.6],
        [-7.6],
    ];
    array.store_array_subset(&[0..8, 6..7], ndarray_subset)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    let ndarray_chunk_subset: Array2<f32> = array![[-7.4, -7.5, -7.6, -7.7],];
    array.store_chunk_subset(
        // chunk indices
        &[1, 1],
        // subset within chunk
        &[3..4, 0..4],
        ndarray_chunk_subset,
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.erase_chunk(&[0, 0])?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.retrieve_chunks(&chunks)?;
    println!("retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.retrieve_array_subset(&subset)?;
    println!("retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

pub fn retrieve_chunks_elements<T: ElementOwned>( &self, chunks: &dyn ArraySubsetTraits, ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_chunks::<Vec<T>>() instead

Read and decode the chunks at chunks into a vector of their elements.

Errors

Returns an ArrayError if any chunk indices in chunks are invalid or an error condition in Array::retrieve_chunks_opt.

Panics

Panics if the number of array elements in the chunks exceeds usize::MAX.

pub fn retrieve_chunks_ndarray<T: ElementOwned>( &self, chunks: &dyn ArraySubsetTraits, ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_chunks::<ndarray::ArrayD<T>>() instead

Available on crate feature ndarray only.

Read and decode the chunks at chunks into an ndarray::ArrayD.

Errors

Returns an ArrayError if any chunk indices in chunks are invalid or an error condition in Array::retrieve_chunks_elements_opt.

Panics

Panics if the number of array elements in the chunks exceeds usize::MAX.

pub fn retrieve_chunk_subset<T: FromArrayBytes>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, ) -> Result<T, ArrayError>

Read and decode the chunk_subset of the chunk at chunk_indices into its bytes.

Errors

Returns an ArrayError if:

• the chunk indices are invalid,
• the chunk subset is invalid,
• there is a codec decoding error, or
• an underlying store error.

Panics

Will panic if the number of elements in chunk_subset is usize::MAX or larger.

pub fn retrieve_chunk_subset_elements<T: ElementOwned>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_chunk_subset::<Vec<T>>() instead

Read and decode the chunk_subset of the chunk at chunk_indices into its elements.

Errors

Returns an ArrayError if:

• the chunk indices are invalid,
• the chunk subset is invalid,
• there is a codec decoding error, or
• an underlying store error.

pub fn retrieve_chunk_subset_ndarray<T: ElementOwned>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_chunk_subset::<ndarray::ArrayD<T>>() instead

Available on crate feature ndarray only.

Read and decode the chunk_subset of the chunk at chunk_indices into an ndarray::ArrayD.

Errors

Returns an ArrayError if:

• the chunk indices are invalid,
• the chunk subset is invalid,
• there is a codec decoding error, or
• an underlying store error.

Panics

Will panic if the number of elements in chunk_subset is usize::MAX or larger.

pub fn retrieve_array_subset<T: FromArrayBytes>( &self, array_subset: &dyn ArraySubsetTraits, ) -> Result<T, ArrayError>

Read and decode the array_subset of array into its bytes.

Out-of-bounds elements will have the fill value.

Errors

Returns an ArrayError if:

• the array_subset dimensionality does not match the chunk grid dimensionality,
• there is a codec decoding error, or
• an underlying store error.

Panics

Panics if attempting to reference a byte beyond usize::MAX.

Examples found in repository

examples/custom_data_type_variable_size.rs (line 191)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let array = ArrayBuilder::new(
        vec![4, 1], // array shape
        vec![3, 1], // regular chunk shape
        Arc::new(CustomDataTypeVariableSize),
        [],
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data = [
        CustomDataTypeVariableSizeElement::from(Some(1.0)),
        CustomDataTypeVariableSizeElement::from(None),
        CustomDataTypeVariableSizeElement::from(Some(3.0)),
    ];
    array.store_chunk(&[0, 0], &data).unwrap();

    let data: Vec<CustomDataTypeVariableSizeElement> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    assert_eq!(data[0], CustomDataTypeVariableSizeElement::from(Some(1.0)));
    assert_eq!(data[1], CustomDataTypeVariableSizeElement::from(None));
    assert_eq!(data[2], CustomDataTypeVariableSizeElement::from(Some(3.0)));
    assert_eq!(data[3], CustomDataTypeVariableSizeElement::from(None));

    println!("{data:#?}");
}

examples/custom_data_type_fixed_size.rs (line 314)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let fill_value = CustomDataTypeFixedSizeElement { x: 1, y: 2.3 };
    let array = ArrayBuilder::new(
        vec![4, 1], // array shape
        vec![2, 1], // regular chunk shape
        Arc::new(CustomDataTypeFixedSize),
        FillValue::new(fill_value.to_ne_bytes().to_vec()),
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data = [
        CustomDataTypeFixedSizeElement { x: 3, y: 4.5 },
        CustomDataTypeFixedSizeElement { x: 6, y: 7.8 },
    ];
    array.store_chunk(&[0, 0], &data).unwrap();

    let data: Vec<CustomDataTypeFixedSizeElement> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    assert_eq!(data[0], CustomDataTypeFixedSizeElement { x: 3, y: 4.5 });
    assert_eq!(data[1], CustomDataTypeFixedSizeElement { x: 6, y: 7.8 });
    assert_eq!(data[2], CustomDataTypeFixedSizeElement { x: 1, y: 2.3 });
    assert_eq!(data[3], CustomDataTypeFixedSizeElement { x: 1, y: 2.3 });

    println!("{data:#?}");
}

examples/custom_data_type_uint12.rs (line 229)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let fill_value = CustomDataTypeUInt12Element::try_from(15).unwrap();
    let array = ArrayBuilder::new(
        vec![4096, 1], // array shape
        vec![5, 1],    // regular chunk shape
        Arc::new(CustomDataTypeUInt12),
        FillValue::new(fill_value.into_le_bytes().to_vec()),
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .array_to_bytes_codec(Arc::new(zarrs::array::codec::PackBitsCodec::default()))
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data: Vec<CustomDataTypeUInt12Element> = (0..4096)
        .map(|i| CustomDataTypeUInt12Element::try_from(i).unwrap())
        .collect();

    array
        .store_array_subset(&array.subset_all(), &data)
        .unwrap();

    let mut data: Vec<CustomDataTypeUInt12Element> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    for (i, d) in data.drain(0..4096).enumerate() {
        let element = CustomDataTypeUInt12Element::try_from(i as u64).unwrap();
        assert_eq!(d, element);
        let element_pd: Vec<CustomDataTypeUInt12Element> = array
            .retrieve_array_subset(&[(i as u64)..i as u64 + 1, 0..1])
            .unwrap();
        assert_eq!(element_pd[0], element);
    }
}

examples/custom_data_type_float8_e3m4.rs (line 240)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let fill_value = CustomDataTypeFloat8e3m4Element::from(1.23);
    let array = ArrayBuilder::new(
        vec![6, 1], // array shape
        vec![5, 1], // regular chunk shape
        Arc::new(CustomDataTypeFloat8e3m4),
        FillValue::new(fill_value.into_ne_bytes().to_vec()),
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data = [
        CustomDataTypeFloat8e3m4Element::from(2.34),
        CustomDataTypeFloat8e3m4Element::from(3.45),
        CustomDataTypeFloat8e3m4Element::from(f32::INFINITY),
        CustomDataTypeFloat8e3m4Element::from(f32::NEG_INFINITY),
        CustomDataTypeFloat8e3m4Element::from(f32::NAN),
    ];
    array.store_chunk(&[0, 0], &data).unwrap();

    let data: Vec<CustomDataTypeFloat8e3m4Element> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    for f in &data {
        println!(
            "float8_e3m4: {:08b} f32: {}",
            f.into_ne_bytes()[0],
            f.into_f32()
        );
    }

    assert_eq!(data[0], CustomDataTypeFloat8e3m4Element::from(2.34));
    assert_eq!(data[1], CustomDataTypeFloat8e3m4Element::from(3.45));
    assert_eq!(
        data[2],
        CustomDataTypeFloat8e3m4Element::from(f32::INFINITY)
    );
    assert_eq!(
        data[3],
        CustomDataTypeFloat8e3m4Element::from(f32::NEG_INFINITY)
    );
    assert_eq!(data[4], CustomDataTypeFloat8e3m4Element::from(f32::NAN));
    assert_eq!(data[5], CustomDataTypeFloat8e3m4Element::from(1.23));
}

examples/custom_data_type_uint4.rs (line 232)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let fill_value = CustomDataTypeUInt4Element::try_from(15).unwrap();
    let array = ArrayBuilder::new(
        vec![6, 1], // array shape
        vec![5, 1], // regular chunk shape
        Arc::new(CustomDataTypeUInt4),
        FillValue::new(fill_value.into_ne_bytes().to_vec()),
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .array_to_bytes_codec(Arc::new(zarrs::array::codec::PackBitsCodec::default()))
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data = [
        CustomDataTypeUInt4Element::try_from(1).unwrap(),
        CustomDataTypeUInt4Element::try_from(2).unwrap(),
        CustomDataTypeUInt4Element::try_from(3).unwrap(),
        CustomDataTypeUInt4Element::try_from(4).unwrap(),
        CustomDataTypeUInt4Element::try_from(5).unwrap(),
    ];
    array.store_chunk(&[0, 0], &data).unwrap();

    let data: Vec<CustomDataTypeUInt4Element> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    for f in &data {
        println!("uint4: {:08b} u8: {}", f.into_u8(), f.into_u8());
    }

    assert_eq!(data[0], CustomDataTypeUInt4Element::try_from(1).unwrap());
    assert_eq!(data[1], CustomDataTypeUInt4Element::try_from(2).unwrap());
    assert_eq!(data[2], CustomDataTypeUInt4Element::try_from(3).unwrap());
    assert_eq!(data[3], CustomDataTypeUInt4Element::try_from(4).unwrap());
    assert_eq!(data[4], CustomDataTypeUInt4Element::try_from(5).unwrap());
    assert_eq!(data[5], CustomDataTypeUInt4Element::try_from(15).unwrap());

    let data: Vec<CustomDataTypeUInt4Element> = array.retrieve_array_subset(&[1..3, 0..1]).unwrap();
    assert_eq!(data[0], CustomDataTypeUInt4Element::try_from(2).unwrap());
    assert_eq!(data[1], CustomDataTypeUInt4Element::try_from(3).unwrap());
}

examples/data_type_optional_nested.rs (line 61)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create an in-memory store
    // let store = Arc::new(zarrs::filesystem::FilesystemStore::new(
    //     "zarrs/tests/data/v3/array_optional_nested.zarr",
    // )?);
    let store = Arc::new(zarrs::storage::store::MemoryStore::new());

    // Build the codec chains for the optional codec
    let array = ArrayBuilder::new(
        vec![4, 4],                                     // 4x4 array
        vec![2, 2],                                     // 2x2 chunks
        data_type::uint8().to_optional().to_optional(), // Optional optional uint8 => Option<Option<u8>>
        FillValue::new_optional_null().into_optional(), // Fill value => Some(None)
    )
    .dimension_names(["y", "x"].into())
    .attributes(
        serde_json::json!({
            "description": r#"A 4x4 array of optional optional uint8 values with some missing data.
The fill value is null on the inner optional layer, i.e. Some(None).
N marks missing (`None`=`null`) values. SN marks `Some(None)`=`[null]` values:
  N  SN   2   3 
  N   5   N   7 
 SN  SN   N   N 
 SN  SN   N   N"#,
        })
        .as_object()
        .unwrap()
        .clone(),
    )
    .build(store.clone(), "/array")?;
    array.store_metadata_opt(
        &zarrs::array::ArrayMetadataOptions::default().with_include_zarrs_metadata(false),
    )?;

    println!("Array metadata:\n{}", array.metadata().to_string_pretty());

    // Create some data with missing values
    let data = ndarray::array![
        [None, Some(None), Some(Some(2u8)), Some(Some(3u8))],
        [None, Some(Some(5u8)), None, Some(Some(7u8))],
        [Some(None), Some(None), None, None],
        [Some(None), Some(None), None, None],
    ]
    .into_dyn();

    // Write the data
    array.store_array_subset(&array.subset_all(), data.clone())?;
    println!("Data written to array.");

    // Read back the data
    let data_read: ArrayD<Option<Option<u8>>> = array.retrieve_array_subset(&array.subset_all())?;

    // Verify data integrity
    assert_eq!(data, data_read);

    // Display the data in a grid format
    println!(
        "Data grid. N marks missing (`None`=`null`) values. SN marks `Some(None)`=`[null]` values"
    );
    println!("    0   1   2   3");
    for y in 0..4 {
        print!("{} ", y);
        for x in 0..4 {
            match data_read[[y, x]] {
                Some(Some(value)) => print!("{:3} ", value),
                Some(None) => print!(" SN "),
                None => print!("  N "),
            }
        }
        println!();
    }
    Ok(())
}

Additional examples can be found in:

• examples/sync_http_array_read.rs
• examples/array_write_read_string.rs
• examples/rectangular_array_write_read.rs
• examples/array_write_read.rs
• examples/sharded_array_write_read.rs
• examples/data_type_optional.rs
• examples/array_write_read_ndarray.rs

pub fn retrieve_array_subset_into( &self, array_subset: &dyn ArraySubsetTraits, output_target: ArrayBytesDecodeIntoTarget<'_>, ) -> Result<(), ArrayError>

Read and decode the array_subset of array into a preallocated output_target.

Only supports fixed-length data types (including optional types with fixed inner types).

Out-of-bounds elements will have the fill value.

Errors

Returns an ArrayError if:

• the array_subset dimensionality does not match the chunk grid dimensionality,
• the data type is variable-length,
• the number of elements in output_target does not match array_subset,
• there is a codec decoding error, or
• an underlying store error.

pub fn retrieve_array_subset_elements<T: ElementOwned>( &self, array_subset: &dyn ArraySubsetTraits, ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_array_subset::<Vec<T>>() instead

Read and decode the array_subset of array into a vector of its elements.

Errors

Returns an ArrayError if:

• the size of T does not match the data type size,
• the decoded bytes cannot be transmuted,
• an array subset is invalid or out of bounds of the array,
• there is a codec decoding error, or
• an underlying store error.

pub fn retrieve_array_subset_ndarray<T: ElementOwned>( &self, array_subset: &dyn ArraySubsetTraits, ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_array_subset::<ndarray::ArrayD<T>>() instead

Available on crate feature ndarray only.

Read and decode the array_subset of array into an ndarray::ArrayD.

Errors

Returns an ArrayError if:

• an array subset is invalid or out of bounds of the array,
• there is a codec decoding error, or
• an underlying store error.

Panics

Will panic if any dimension in chunk_subset is usize::MAX or larger.

pub fn partial_decoder( &self, chunk_indices: &[u64], ) -> Result<Arc<dyn ArrayPartialDecoderTraits>, ArrayError>

Initialises a partial decoder for the chunk at chunk_indices.

Errors

Returns an ArrayError if initialisation of the partial decoder fails.

Examples found in repository

examples/sharded_array_write_read.rs (line 128)

fn sharded_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use rayon::prelude::{IntoParallelIterator, ParallelIterator};
    use zarrs::array::{ArraySubset, codec, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/sharded_array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    // Create an array
    let array_path = "/group/array";
    let subchunk_shape = vec![4, 4];
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 8], // chunk (shard) shape
        data_type::uint16(),
        0u16,
    )
    .subchunk_shape(subchunk_shape.clone())
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(codec::GzipCodec::new(5)?),
    ])
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    // The array metadata is
    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Use default codec options (concurrency etc)
    let options = CodecOptions::default();

    // Write some shards (in parallel)
    (0..2).into_par_iter().try_for_each(|s| {
        let chunk_grid = array.chunk_grid();
        let chunk_indices = vec![s, 0];
        if let Some(chunk_shape) = chunk_grid.chunk_shape(&chunk_indices)? {
            let chunk_array = ndarray::ArrayD::<u16>::from_shape_fn(
                chunk_shape
                    .iter()
                    .map(|u| u.get() as usize)
                    .collect::<Vec<_>>(),
                |ij| {
                    (s * chunk_shape[0].get() * chunk_shape[1].get()
                        + ij[0] as u64 * chunk_shape[1].get()
                        + ij[1] as u64) as u16
                },
            );
            array.store_chunk(&chunk_indices, chunk_array)
        } else {
            Err(zarrs::array::ArrayError::InvalidChunkGridIndicesError(
                chunk_indices.to_vec(),
            ))
        }
    })?;

    // Read the whole array
    let data_all: ArrayD<u16> = array.retrieve_array_subset(&array.subset_all())?;
    println!("The whole array is:\n{data_all}\n");

    // Read a shard back from the store
    let shard_indices = vec![1, 0];
    let data_shard: ArrayD<u16> = array.retrieve_chunk(&shard_indices)?;
    println!("Shard [1,0] is:\n{data_shard}\n");

    // Read a subchunk from the store
    let subset_chunk_1_0 = ArraySubset::new_with_ranges(&[4..8, 0..4]);
    let data_chunk: ArrayD<u16> = array.retrieve_array_subset(&subset_chunk_1_0)?;
    println!("Chunk [1,0] is:\n{data_chunk}\n");

    // Read the central 4x2 subset of the array
    let subset_4x2 = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_4x2: ArrayD<u16> = array.retrieve_array_subset(&subset_4x2)?;
    println!("The middle 4x2 subset is:\n{data_4x2}\n");

    // Decode subchunks
    // In some cases, it might be preferable to decode subchunks in a shard directly.
    // If using the partial decoder, then the shard index will only be read once from the store.
    let partial_decoder = array.partial_decoder(&[0, 0])?;
    println!("Decoded subchunks:");
    for subchunk_subset in [
        ArraySubset::new_with_start_shape(vec![0, 0], subchunk_shape.clone())?,
        ArraySubset::new_with_start_shape(vec![0, 4], subchunk_shape.clone())?,
    ] {
        println!("{subchunk_subset}");
        let decoded_subchunk_bytes = partial_decoder.partial_decode(&subchunk_subset, &options)?;
        let ndarray = bytes_to_ndarray::<u16>(
            &subchunk_shape,
            decoded_subchunk_bytes.into_fixed()?.into_owned(),
        )?;
        println!("{ndarray}\n");
    }

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("The Zarr hierarchy tree is:\n{}", tree);

    println!(
        "The keys in the store are:\n[{}]",
        store.list().unwrap_or_default().iter().format(", ")
    );

    Ok(())
}

pub fn retrieve_chunk_if_exists_opt<T: FromArrayBytes>( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<Option<T>, ArrayError>

Explicit options version of retrieve_chunk_if_exists.

pub fn retrieve_chunk_opt<T: FromArrayBytes>( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<T, ArrayError>

Explicit options version of retrieve_chunk.

pub fn retrieve_chunk_elements_if_exists_opt<T: ElementOwned>( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<Option<Vec<T>>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_chunk_if_exists_opt::<Vec<T>>() instead

Explicit options version of retrieve_chunk_elements_if_exists.

pub fn retrieve_chunk_elements_opt<T: ElementOwned>( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_chunk_opt::<Vec<T>>() instead

Explicit options version of retrieve_chunk_elements.

pub fn retrieve_chunk_ndarray_if_exists_opt<T: ElementOwned>( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<Option<ArrayD<T>>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_chunk_if_exists_opt::<ndarray::ArrayD<T>>() instead

Available on crate feature ndarray only.

Explicit options version of retrieve_chunk_ndarray_if_exists.

pub fn retrieve_chunk_ndarray_opt<T: ElementOwned>( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_chunk_opt::<ndarray::ArrayD<T>>() instead

Available on crate feature ndarray only.

Explicit options version of retrieve_chunk_ndarray.

pub fn retrieve_chunks_opt<T: FromArrayBytes>( &self, chunks: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<T, ArrayError>

Explicit options version of retrieve_chunks.

pub fn retrieve_chunks_elements_opt<T: ElementOwned>( &self, chunks: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_chunks_opt::<Vec<T>>() instead

Explicit options version of retrieve_chunks_elements.

pub fn retrieve_chunks_ndarray_opt<T: ElementOwned>( &self, chunks: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_chunks_opt::<ndarray::ArrayD<T>>() instead

Available on crate feature ndarray only.

Explicit options version of retrieve_chunks_ndarray.

pub fn retrieve_array_subset_opt<T: FromArrayBytes>( &self, array_subset: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<T, ArrayError>

Explicit options version of retrieve_array_subset.

pub fn retrieve_array_subset_into_opt( &self, array_subset: &dyn ArraySubsetTraits, output_target: ArrayBytesDecodeIntoTarget<'_>, options: &CodecOptions, ) -> Result<(), ArrayError>

Explicit options version of retrieve_array_subset_into.

pub fn retrieve_array_subset_elements_opt<T: ElementOwned>( &self, array_subset: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_array_subset_opt::<Vec<T>>() instead

Explicit options version of retrieve_array_subset_elements.

pub fn retrieve_array_subset_ndarray_opt<T: ElementOwned>( &self, array_subset: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_array_subset_opt::<ndarray::ArrayD<T>>() instead

Available on crate feature ndarray only.

Explicit options version of retrieve_array_subset_ndarray.

pub fn retrieve_chunk_subset_opt<T: FromArrayBytes>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<T, ArrayError>

Explicit options version of retrieve_chunk_subset.

pub fn retrieve_chunk_subset_elements_opt<T: ElementOwned>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_chunk_subset_opt::<Vec<T>>() instead

Explicit options version of retrieve_chunk_subset_elements.

pub fn retrieve_chunk_subset_ndarray_opt<T: ElementOwned>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_chunk_subset_opt::<ndarray::ArrayD<T>>() instead

Available on crate feature ndarray only.

Explicit options version of retrieve_chunk_subset_ndarray.

pub fn partial_decoder_opt( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<Arc<dyn ArrayPartialDecoderTraits>, ArrayError>

Explicit options version of partial_decoder.

impl<TStorage: ?Sized + WritableStorageTraits + 'static> Array<TStorage>

pub fn store_metadata(&self) -> Result<(), StorageError>

Store metadata with default ArrayMetadataOptions.

The metadata is created with Array::metadata_opt.

Errors

Returns StorageError if there is an underlying store error.

Examples found in repository

examples/zarr_v2_to_v3.rs (line 81)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let store = Arc::new(zarrs_storage::store::MemoryStore::new());

    let serde_json::Value::Object(attributes) = serde_json::json!({
        "foo": "bar",
        "baz": 42,
    }) else {
        unreachable!()
    };

    // Create a Zarr V2 group
    let group_metadata: GroupMetadata = GroupMetadataV2::new()
        .with_attributes(attributes.clone())
        .into();
    let group = Group::new_with_metadata(store.clone(), "/group", group_metadata)?;

    // Store the metadata as V2 and V3
    let convert_group_metadata_to_v3 =
        GroupMetadataOptions::default().with_metadata_convert_version(MetadataConvertVersion::V3);
    group.store_metadata()?;
    group.store_metadata_opt(&convert_group_metadata_to_v3)?;
    println!(
        "group/.zgroup (Zarr V2 group metadata):\n{}\n",
        key_to_str(&store, "group/.zgroup")?
    );
    println!(
        "group/.zattrs (Zarr V2 group attributes):\n{}\n",
        key_to_str(&store, "group/.zattrs")?
    );
    println!(
        "group/zarr.json (Zarr V3 equivalent group metadata/attributes):\n{}\n",
        key_to_str(&store, "group/zarr.json")?
    );
    // println!(
    //     "The equivalent Zarr V3 group metadata is\n{}\n",
    //     group.metadata_opt(&convert_group_metadata_to_v3).to_string_pretty()
    // );

    // Create a Zarr V2 array
    let array_metadata = ArrayMetadataV2::new(
        vec![10, 10],
        vec![NonZeroU64::new(5).unwrap(); 2],
        ">f4".into(), // big endian float32
        FillValueMetadata::from(f32::NAN),
        None,
        None,
    )
    .with_dimension_separator(ChunkKeySeparator::Slash)
    .with_order(ArrayMetadataV2Order::F)
    .with_attributes(attributes.clone());
    let array = zarrs::array::Array::new_with_metadata(
        store.clone(),
        "/group/array",
        array_metadata.into(),
    )?;

    // Store the metadata as V2 and V3
    let convert_array_metadata_to_v3 =
        ArrayMetadataOptions::default().with_metadata_convert_version(MetadataConvertVersion::V3);
    array.store_metadata()?;
    array.store_metadata_opt(&convert_array_metadata_to_v3)?;
    println!(
        "group/array/.zarray (Zarr V2 array metadata):\n{}\n",
        key_to_str(&store, "group/array/.zarray")?
    );
    println!(
        "group/array/.zattrs (Zarr V2 array attributes):\n{}\n",
        key_to_str(&store, "group/array/.zattrs")?
    );
    println!(
        "group/array/zarr.json (Zarr V3 equivalent array metadata/attributes):\n{}\n",
        key_to_str(&store, "group/array/zarr.json")?
    );
    // println!(
    //     "The equivalent Zarr V3 array metadata is\n{}\n",
    //     array.metadata_opt(&convert_array_metadata_to_v3).to_string_pretty()
    // );

    array.store_chunk(&[0, 1], &[0.0f32; 5 * 5])?;

    // Print the keys in the store
    println!("The store contains keys:");
    for key in store.list()? {
        println!("  {}", key);
    }

    Ok(())
}

examples/array_write_read_string.rs (line 67)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArrayBytes, data_type};
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![4, 4], // array shape
        vec![2, 2], // regular chunk shape
        data_type::string(),
        "_",
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    array.store_chunk(
        &[0, 0],
        ArrayD::<&str>::from_shape_vec(vec![2, 2], vec!["a", "bb", "ccc", "dddd"]).unwrap(),
    )?;
    array.store_chunk(
        &[0, 1],
        ArrayD::<&str>::from_shape_vec(vec![2, 2], vec!["4444", "333", "22", "1"]).unwrap(),
    )?;
    let subset_all = array.subset_all();
    let data_all: ArrayD<String> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    let ndarray_subset: Array2<&str> = array![["!", "@@"], ["###", "$$$$"]];
    array.store_array_subset(&[1..3, 1..3], ndarray_subset)?;
    let data_all: ArrayD<String> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [1..3, 1..3]:\nndarray::ArrayD<String>\n{data_all}");

    // Retrieve bytes directly, convert into a single string allocation, create a &str ndarray
    // TODO: Add a convenience function for this?
    let data_all: ArrayBytes = array.retrieve_array_subset(&subset_all)?;
    let (bytes, offsets) = data_all.into_variable()?.into_parts();
    let string = String::from_utf8(bytes.into_owned())?;
    let elements = offsets
        .iter()
        .tuple_windows()
        .map(|(&curr, &next)| &string[curr..next])
        .collect::<Vec<&str>>();
    let ndarray = ArrayD::<&str>::from_shape_vec(subset_all.shape_usize(), elements)?;
    println!("ndarray::ArrayD<&str>:\n{ndarray}");

    Ok(())
}

examples/rectangular_array_write_read.rs (line 86)

fn rectangular_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use rayon::prelude::{IntoParallelIterator, ParallelIterator};
    use zarrs::array::{ArraySubset, ZARR_NAN_F32, codec, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        MetadataV3::new_with_configuration(
            "rectangular",
            RectangularChunkGridConfiguration {
                chunk_shape: vec![
                    vec![
                        NonZeroU64::new(1).unwrap(),
                        NonZeroU64::new(2).unwrap(),
                        NonZeroU64::new(3).unwrap(),
                        NonZeroU64::new(2).unwrap(),
                    ]
                    .into(),
                    NonZeroU64::new(4).unwrap().into(),
                ], // chunk sizes
            },
        ),
        data_type::float32(),
        ZARR_NAN_F32,
    )
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(codec::GzipCodec::new(5)?),
    ])
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    // Write some chunks (in parallel)
    (0..4).into_par_iter().try_for_each(|i| {
        let chunk_grid = array.chunk_grid();
        let chunk_indices = vec![i, 0];
        if let Some(chunk_shape) = chunk_grid.chunk_shape(&chunk_indices)? {
            let chunk_array = ndarray::ArrayD::<f32>::from_elem(
                chunk_shape
                    .iter()
                    .map(|u| u.get() as usize)
                    .collect::<Vec<_>>(),
                i as f32,
            );
            array.store_chunk(&chunk_indices, chunk_array)
        } else {
            Err(zarrs::array::ArrayError::InvalidChunkGridIndicesError(
                chunk_indices.to_vec(),
            ))
        }
    })?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write a subset spanning multiple chunks, including updating chunks already written
    array.store_array_subset(
        &[3..6, 3..6], // start
        ndarray::ArrayD::<f32>::from_shape_vec(
            vec![3, 3],
            vec![0.1f32, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9],
        )?,
    )?;

    // Store elements directly, in this case set the 7th column to 123.0
    array.store_array_subset(&[0..8, 6..7], &[123.0f32; 8])?;

    // Store elements directly in a chunk, in this case set the last row of the bottom right chunk
    array.store_chunk_subset(
        // chunk indices
        &[3, 1],
        // subset within chunk
        &[1..2, 0..4],
        &[-4.0f32; 4],
    )?;

    // Read the whole array
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&array.subset_all())?;
    println!("The whole array is:\n{data_all}\n");

    // Read a chunk back from the store
    let chunk_indices = vec![1, 0];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("Chunk [1,0] is:\n{data_chunk}\n");

    // Read the central 4x2 subset of the array
    let subset_4x2 = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_4x2: ArrayD<f32> = array.retrieve_array_subset(&subset_4x2)?;
    println!("The middle 4x2 subset is:\n{data_4x2}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("The Zarr hierarchy tree is:\n{tree}");

    Ok(())
}

examples/array_write_read.rs (line 68)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    (0..2).into_par_iter().try_for_each(|i| {
        let chunk_indices: Vec<u64> = vec![0, i];
        let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
            zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
        })?;
        array.store_chunk(
            &chunk_indices,
            vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
        )
    })?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array.store_chunks(
        &[1..2, 0..2],
        &[
            //
            1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            //
            1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
        ],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array.store_array_subset(
        &[3..6, 3..6],
        &[-3.3f32, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array.store_array_subset(
        &[0..8, 6..7],
        &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array.store_chunk_subset(
        // chunk indices
        &[1, 1],
        // subset within chunk
        &[3..4, 0..4],
        &[-7.4f32, -7.5, -7.6, -7.7],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.erase_chunk(&[0, 0])?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.retrieve_chunks(&chunks)?;
    println!("retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.retrieve_array_subset(&subset)?;
    println!("retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

examples/sharded_array_write_read.rs (line 71)

fn sharded_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use rayon::prelude::{IntoParallelIterator, ParallelIterator};
    use zarrs::array::{ArraySubset, codec, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/sharded_array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    // Create an array
    let array_path = "/group/array";
    let subchunk_shape = vec![4, 4];
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 8], // chunk (shard) shape
        data_type::uint16(),
        0u16,
    )
    .subchunk_shape(subchunk_shape.clone())
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(codec::GzipCodec::new(5)?),
    ])
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    // The array metadata is
    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Use default codec options (concurrency etc)
    let options = CodecOptions::default();

    // Write some shards (in parallel)
    (0..2).into_par_iter().try_for_each(|s| {
        let chunk_grid = array.chunk_grid();
        let chunk_indices = vec![s, 0];
        if let Some(chunk_shape) = chunk_grid.chunk_shape(&chunk_indices)? {
            let chunk_array = ndarray::ArrayD::<u16>::from_shape_fn(
                chunk_shape
                    .iter()
                    .map(|u| u.get() as usize)
                    .collect::<Vec<_>>(),
                |ij| {
                    (s * chunk_shape[0].get() * chunk_shape[1].get()
                        + ij[0] as u64 * chunk_shape[1].get()
                        + ij[1] as u64) as u16
                },
            );
            array.store_chunk(&chunk_indices, chunk_array)
        } else {
            Err(zarrs::array::ArrayError::InvalidChunkGridIndicesError(
                chunk_indices.to_vec(),
            ))
        }
    })?;

    // Read the whole array
    let data_all: ArrayD<u16> = array.retrieve_array_subset(&array.subset_all())?;
    println!("The whole array is:\n{data_all}\n");

    // Read a shard back from the store
    let shard_indices = vec![1, 0];
    let data_shard: ArrayD<u16> = array.retrieve_chunk(&shard_indices)?;
    println!("Shard [1,0] is:\n{data_shard}\n");

    // Read a subchunk from the store
    let subset_chunk_1_0 = ArraySubset::new_with_ranges(&[4..8, 0..4]);
    let data_chunk: ArrayD<u16> = array.retrieve_array_subset(&subset_chunk_1_0)?;
    println!("Chunk [1,0] is:\n{data_chunk}\n");

    // Read the central 4x2 subset of the array
    let subset_4x2 = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_4x2: ArrayD<u16> = array.retrieve_array_subset(&subset_4x2)?;
    println!("The middle 4x2 subset is:\n{data_4x2}\n");

    // Decode subchunks
    // In some cases, it might be preferable to decode subchunks in a shard directly.
    // If using the partial decoder, then the shard index will only be read once from the store.
    let partial_decoder = array.partial_decoder(&[0, 0])?;
    println!("Decoded subchunks:");
    for subchunk_subset in [
        ArraySubset::new_with_start_shape(vec![0, 0], subchunk_shape.clone())?,
        ArraySubset::new_with_start_shape(vec![0, 4], subchunk_shape.clone())?,
    ] {
        println!("{subchunk_subset}");
        let decoded_subchunk_bytes = partial_decoder.partial_decode(&subchunk_subset, &options)?;
        let ndarray = bytes_to_ndarray::<u16>(
            &subchunk_shape,
            decoded_subchunk_bytes.into_fixed()?.into_owned(),
        )?;
        println!("{ndarray}\n");
    }

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("The Zarr hierarchy tree is:\n{}", tree);

    println!(
        "The keys in the store are:\n[{}]",
        store.list().unwrap_or_default().iter().format(", ")
    );

    Ok(())
}

examples/array_write_read_ndarray.rs (line 68)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    (0..2).into_par_iter().try_for_each(|i| {
        let chunk_indices: Vec<u64> = vec![0, i];
        let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
            zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
        })?;
        array.store_chunk(
            &chunk_indices,
            ArrayD::<f32>::from_shape_vec(
                chunk_subset.shape_usize(),
                vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
            )
            .unwrap(),
        )
    })?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    let ndarray_chunks: Array2<f32> = array![
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
    ];
    array.store_chunks(&[1..2, 0..2], ndarray_chunks)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    let ndarray_subset: Array2<f32> =
        array![[-3.3, -3.4, -3.5,], [-4.3, -4.4, -4.5,], [-5.3, -5.4, -5.5],];
    array.store_array_subset(&[3..6, 3..6], ndarray_subset)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    let ndarray_subset: Array2<f32> = array![
        [-0.6],
        [-1.6],
        [-2.6],
        [-3.6],
        [-4.6],
        [-5.6],
        [-6.6],
        [-7.6],
    ];
    array.store_array_subset(&[0..8, 6..7], ndarray_subset)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    let ndarray_chunk_subset: Array2<f32> = array![[-7.4, -7.5, -7.6, -7.7],];
    array.store_chunk_subset(
        // chunk indices
        &[1, 1],
        // subset within chunk
        &[3..4, 0..4],
        ndarray_chunk_subset,
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.erase_chunk(&[0, 0])?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.retrieve_chunks(&chunks)?;
    println!("retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.retrieve_array_subset(&subset)?;
    println!("retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

pub fn store_metadata_opt( &self, options: &ArrayMetadataOptions, ) -> Result<(), StorageError>

Store metadata with non-default ArrayMetadataOptions.

The metadata is created with Array::metadata_opt.

Errors

Returns StorageError if there is an underlying store error.

Examples found in repository

examples/data_type_optional_nested.rs (lines 41-43)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create an in-memory store
    // let store = Arc::new(zarrs::filesystem::FilesystemStore::new(
    //     "zarrs/tests/data/v3/array_optional_nested.zarr",
    // )?);
    let store = Arc::new(zarrs::storage::store::MemoryStore::new());

    // Build the codec chains for the optional codec
    let array = ArrayBuilder::new(
        vec![4, 4],                                     // 4x4 array
        vec![2, 2],                                     // 2x2 chunks
        data_type::uint8().to_optional().to_optional(), // Optional optional uint8 => Option<Option<u8>>
        FillValue::new_optional_null().into_optional(), // Fill value => Some(None)
    )
    .dimension_names(["y", "x"].into())
    .attributes(
        serde_json::json!({
            "description": r#"A 4x4 array of optional optional uint8 values with some missing data.
The fill value is null on the inner optional layer, i.e. Some(None).
N marks missing (`None`=`null`) values. SN marks `Some(None)`=`[null]` values:
  N  SN   2   3 
  N   5   N   7 
 SN  SN   N   N 
 SN  SN   N   N"#,
        })
        .as_object()
        .unwrap()
        .clone(),
    )
    .build(store.clone(), "/array")?;
    array.store_metadata_opt(
        &zarrs::array::ArrayMetadataOptions::default().with_include_zarrs_metadata(false),
    )?;

    println!("Array metadata:\n{}", array.metadata().to_string_pretty());

    // Create some data with missing values
    let data = ndarray::array![
        [None, Some(None), Some(Some(2u8)), Some(Some(3u8))],
        [None, Some(Some(5u8)), None, Some(Some(7u8))],
        [Some(None), Some(None), None, None],
        [Some(None), Some(None), None, None],
    ]
    .into_dyn();

    // Write the data
    array.store_array_subset(&array.subset_all(), data.clone())?;
    println!("Data written to array.");

    // Read back the data
    let data_read: ArrayD<Option<Option<u8>>> = array.retrieve_array_subset(&array.subset_all())?;

    // Verify data integrity
    assert_eq!(data, data_read);

    // Display the data in a grid format
    println!(
        "Data grid. N marks missing (`None`=`null`) values. SN marks `Some(None)`=`[null]` values"
    );
    println!("    0   1   2   3");
    for y in 0..4 {
        print!("{} ", y);
        for x in 0..4 {
            match data_read[[y, x]] {
                Some(Some(value)) => print!("{:3} ", value),
                Some(None) => print!(" SN "),
                None => print!("  N "),
            }
        }
        println!();
    }
    Ok(())
}

examples/zarr_v2_to_v3.rs (line 82)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let store = Arc::new(zarrs_storage::store::MemoryStore::new());

    let serde_json::Value::Object(attributes) = serde_json::json!({
        "foo": "bar",
        "baz": 42,
    }) else {
        unreachable!()
    };

    // Create a Zarr V2 group
    let group_metadata: GroupMetadata = GroupMetadataV2::new()
        .with_attributes(attributes.clone())
        .into();
    let group = Group::new_with_metadata(store.clone(), "/group", group_metadata)?;

    // Store the metadata as V2 and V3
    let convert_group_metadata_to_v3 =
        GroupMetadataOptions::default().with_metadata_convert_version(MetadataConvertVersion::V3);
    group.store_metadata()?;
    group.store_metadata_opt(&convert_group_metadata_to_v3)?;
    println!(
        "group/.zgroup (Zarr V2 group metadata):\n{}\n",
        key_to_str(&store, "group/.zgroup")?
    );
    println!(
        "group/.zattrs (Zarr V2 group attributes):\n{}\n",
        key_to_str(&store, "group/.zattrs")?
    );
    println!(
        "group/zarr.json (Zarr V3 equivalent group metadata/attributes):\n{}\n",
        key_to_str(&store, "group/zarr.json")?
    );
    // println!(
    //     "The equivalent Zarr V3 group metadata is\n{}\n",
    //     group.metadata_opt(&convert_group_metadata_to_v3).to_string_pretty()
    // );

    // Create a Zarr V2 array
    let array_metadata = ArrayMetadataV2::new(
        vec![10, 10],
        vec![NonZeroU64::new(5).unwrap(); 2],
        ">f4".into(), // big endian float32
        FillValueMetadata::from(f32::NAN),
        None,
        None,
    )
    .with_dimension_separator(ChunkKeySeparator::Slash)
    .with_order(ArrayMetadataV2Order::F)
    .with_attributes(attributes.clone());
    let array = zarrs::array::Array::new_with_metadata(
        store.clone(),
        "/group/array",
        array_metadata.into(),
    )?;

    // Store the metadata as V2 and V3
    let convert_array_metadata_to_v3 =
        ArrayMetadataOptions::default().with_metadata_convert_version(MetadataConvertVersion::V3);
    array.store_metadata()?;
    array.store_metadata_opt(&convert_array_metadata_to_v3)?;
    println!(
        "group/array/.zarray (Zarr V2 array metadata):\n{}\n",
        key_to_str(&store, "group/array/.zarray")?
    );
    println!(
        "group/array/.zattrs (Zarr V2 array attributes):\n{}\n",
        key_to_str(&store, "group/array/.zattrs")?
    );
    println!(
        "group/array/zarr.json (Zarr V3 equivalent array metadata/attributes):\n{}\n",
        key_to_str(&store, "group/array/zarr.json")?
    );
    // println!(
    //     "The equivalent Zarr V3 array metadata is\n{}\n",
    //     array.metadata_opt(&convert_array_metadata_to_v3).to_string_pretty()
    // );

    array.store_chunk(&[0, 1], &[0.0f32; 5 * 5])?;

    // Print the keys in the store
    println!("The store contains keys:");
    for key in store.list()? {
        println!("  {}", key);
    }

    Ok(())
}

examples/data_type_optional.rs (lines 47-49)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create an in-memory store
    // let store = Arc::new(zarrs::filesystem::FilesystemStore::new(
    //     "zarrs/tests/data/v3/array_optional.zarr",
    // )?);
    let store = Arc::new(zarrs::storage::store::MemoryStore::new());

    // Build the codec chains for the optional codec
    let array = ArrayBuilder::new(
        vec![4, 4],                       // 4x4 array
        vec![2, 2],                       // 2x2 chunks
        data_type::uint8().to_optional(), // Optional uint8
        FillValue::new_optional_null(),   // Null fill value: [0]
    )
    .dimension_names(["y", "x"].into())
    .attributes(
        serde_json::json!({
            "description": r#"A 4x4 array of optional uint8 values with some missing data.
N marks missing (`None`=`null`) values:
 0  N  2  3 
 N  5  N  7 
 8  9  N  N 
12  N  N  N"#,
        })
        .as_object()
        .unwrap()
        .clone(),
    )
    .build(store.clone(), "/array")?;
    array.store_metadata_opt(
        &zarrs::array::ArrayMetadataOptions::default().with_include_zarrs_metadata(false),
    )?;

    println!("Array metadata:\n{}", array.metadata().to_string_pretty());

    // Create some data with missing values
    let data = ndarray::array![
        [Some(0u8), None, Some(2u8), Some(3u8)],
        [None, Some(5u8), None, Some(7u8)],
        [Some(8u8), Some(9u8), None, None],
        [Some(12u8), None, None, None],
    ]
    .into_dyn();

    // Write the data
    array.store_array_subset(&array.subset_all(), data.clone())?;

    // Read back the data
    let data_read: ArrayD<Option<u8>> = array.retrieve_array_subset(&array.subset_all())?;

    // Verify data integrity
    assert_eq!(data, data_read);

    // Display the data in a grid format
    println!("Data grid, N marks missing (`None`=`null`) values");
    println!("   0  1  2  3");
    for y in 0..4 {
        print!("{} ", y);
        for x in 0..4 {
            match data_read[[y, x]] {
                Some(value) => print!("{:2} ", value),
                None => print!(" N "),
            }
        }
        println!();
    }

    // Print the raw bytes in all chunks
    println!("Raw bytes in all chunks:");
    let chunk_grid_shape = array.chunk_grid_shape();
    for chunk_y in 0..chunk_grid_shape[0] {
        for chunk_x in 0..chunk_grid_shape[1] {
            let chunk_indices = vec![chunk_y, chunk_x];
            let chunk_key = array.chunk_key(&chunk_indices);
            println!("  Chunk [{}, {}] (key: {}):", chunk_y, chunk_x, chunk_key);

            if let Some(chunk_bytes) = store.get(&chunk_key)? {
                println!("    Size: {} bytes", chunk_bytes.len());

                if chunk_bytes.len() >= 16 {
                    // Parse first 8 bytes as mask size (little-endian u64)
                    let mask_size = u64::from_le_bytes([
                        chunk_bytes[0],
                        chunk_bytes[1],
                        chunk_bytes[2],
                        chunk_bytes[3],
                        chunk_bytes[4],
                        chunk_bytes[5],
                        chunk_bytes[6],
                        chunk_bytes[7],
                    ]) as usize;

                    // Parse second 8 bytes as data size (little-endian u64)
                    let data_size = u64::from_le_bytes([
                        chunk_bytes[8],
                        chunk_bytes[9],
                        chunk_bytes[10],
                        chunk_bytes[11],
                        chunk_bytes[12],
                        chunk_bytes[13],
                        chunk_bytes[14],
                        chunk_bytes[15],
                    ]) as usize;

                    // Display mask size header with raw bytes
                    print!("    Mask size: 0b");
                    for byte in &chunk_bytes[0..8] {
                        print!("{:08b}", byte);
                    }
                    println!(" -> {} bytes", mask_size);

                    // Display data size header with raw bytes
                    print!("    Data size: 0b");
                    for byte in &chunk_bytes[8..16] {
                        print!("{:08b}", byte);
                    }
                    println!(" -> {} bytes", data_size);

                    // Show mask and data sections separately
                    if chunk_bytes.len() >= 16 + mask_size + data_size {
                        let mask_start = 16;
                        let data_start = 16 + mask_size;

                        // Show mask as binary
                        if mask_size > 0 {
                            println!("    Mask (binary):");
                            print!("      ");
                            for byte in &chunk_bytes[mask_start..mask_start + mask_size] {
                                print!("0b{:08b} ", byte);
                            }
                            println!();
                        }

                        // Show data as binary
                        if data_size > 0 {
                            println!("    Data (binary):");
                            print!("      ");
                            for byte in &chunk_bytes[data_start..data_start + data_size] {
                                print!("0b{:08b} ", byte);
                            }
                            println!();
                        }
                    }
                } else {
                    panic!("    Chunk too small to parse headers");
                }
            } else {
                println!("    Chunk missing (fill value chunk)");
            }
        }
    }
    Ok(())
}

pub fn store_chunk<'a>( &self, chunk_indices: &[u64], chunk_data: impl IntoArrayBytes<'a>, ) -> Result<(), ArrayError>

Encode chunk_data and store at chunk_indices.

Use store_chunk_opt to control codec options. A chunk composed entirely of the fill value will not be written to the store.

Errors

Returns an ArrayError if

• chunk_indices are invalid,
• the length of chunk_data is not equal to the expected length (the product of the number of elements in the chunk and the data type size in bytes),
• there is a codec encoding error, or
• an underlying store error.

Examples found in repository

examples/custom_data_type_variable_size.rs (line 188)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let array = ArrayBuilder::new(
        vec![4, 1], // array shape
        vec![3, 1], // regular chunk shape
        Arc::new(CustomDataTypeVariableSize),
        [],
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data = [
        CustomDataTypeVariableSizeElement::from(Some(1.0)),
        CustomDataTypeVariableSizeElement::from(None),
        CustomDataTypeVariableSizeElement::from(Some(3.0)),
    ];
    array.store_chunk(&[0, 0], &data).unwrap();

    let data: Vec<CustomDataTypeVariableSizeElement> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    assert_eq!(data[0], CustomDataTypeVariableSizeElement::from(Some(1.0)));
    assert_eq!(data[1], CustomDataTypeVariableSizeElement::from(None));
    assert_eq!(data[2], CustomDataTypeVariableSizeElement::from(Some(3.0)));
    assert_eq!(data[3], CustomDataTypeVariableSizeElement::from(None));

    println!("{data:#?}");
}

examples/custom_data_type_fixed_size.rs (line 311)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let fill_value = CustomDataTypeFixedSizeElement { x: 1, y: 2.3 };
    let array = ArrayBuilder::new(
        vec![4, 1], // array shape
        vec![2, 1], // regular chunk shape
        Arc::new(CustomDataTypeFixedSize),
        FillValue::new(fill_value.to_ne_bytes().to_vec()),
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data = [
        CustomDataTypeFixedSizeElement { x: 3, y: 4.5 },
        CustomDataTypeFixedSizeElement { x: 6, y: 7.8 },
    ];
    array.store_chunk(&[0, 0], &data).unwrap();

    let data: Vec<CustomDataTypeFixedSizeElement> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    assert_eq!(data[0], CustomDataTypeFixedSizeElement { x: 3, y: 4.5 });
    assert_eq!(data[1], CustomDataTypeFixedSizeElement { x: 6, y: 7.8 });
    assert_eq!(data[2], CustomDataTypeFixedSizeElement { x: 1, y: 2.3 });
    assert_eq!(data[3], CustomDataTypeFixedSizeElement { x: 1, y: 2.3 });

    println!("{data:#?}");
}

examples/custom_data_type_float8_e3m4.rs (line 237)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let fill_value = CustomDataTypeFloat8e3m4Element::from(1.23);
    let array = ArrayBuilder::new(
        vec![6, 1], // array shape
        vec![5, 1], // regular chunk shape
        Arc::new(CustomDataTypeFloat8e3m4),
        FillValue::new(fill_value.into_ne_bytes().to_vec()),
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data = [
        CustomDataTypeFloat8e3m4Element::from(2.34),
        CustomDataTypeFloat8e3m4Element::from(3.45),
        CustomDataTypeFloat8e3m4Element::from(f32::INFINITY),
        CustomDataTypeFloat8e3m4Element::from(f32::NEG_INFINITY),
        CustomDataTypeFloat8e3m4Element::from(f32::NAN),
    ];
    array.store_chunk(&[0, 0], &data).unwrap();

    let data: Vec<CustomDataTypeFloat8e3m4Element> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    for f in &data {
        println!(
            "float8_e3m4: {:08b} f32: {}",
            f.into_ne_bytes()[0],
            f.into_f32()
        );
    }

    assert_eq!(data[0], CustomDataTypeFloat8e3m4Element::from(2.34));
    assert_eq!(data[1], CustomDataTypeFloat8e3m4Element::from(3.45));
    assert_eq!(
        data[2],
        CustomDataTypeFloat8e3m4Element::from(f32::INFINITY)
    );
    assert_eq!(
        data[3],
        CustomDataTypeFloat8e3m4Element::from(f32::NEG_INFINITY)
    );
    assert_eq!(data[4], CustomDataTypeFloat8e3m4Element::from(f32::NAN));
    assert_eq!(data[5], CustomDataTypeFloat8e3m4Element::from(1.23));
}

examples/custom_data_type_uint4.rs (line 229)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let fill_value = CustomDataTypeUInt4Element::try_from(15).unwrap();
    let array = ArrayBuilder::new(
        vec![6, 1], // array shape
        vec![5, 1], // regular chunk shape
        Arc::new(CustomDataTypeUInt4),
        FillValue::new(fill_value.into_ne_bytes().to_vec()),
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .array_to_bytes_codec(Arc::new(zarrs::array::codec::PackBitsCodec::default()))
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data = [
        CustomDataTypeUInt4Element::try_from(1).unwrap(),
        CustomDataTypeUInt4Element::try_from(2).unwrap(),
        CustomDataTypeUInt4Element::try_from(3).unwrap(),
        CustomDataTypeUInt4Element::try_from(4).unwrap(),
        CustomDataTypeUInt4Element::try_from(5).unwrap(),
    ];
    array.store_chunk(&[0, 0], &data).unwrap();

    let data: Vec<CustomDataTypeUInt4Element> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    for f in &data {
        println!("uint4: {:08b} u8: {}", f.into_u8(), f.into_u8());
    }

    assert_eq!(data[0], CustomDataTypeUInt4Element::try_from(1).unwrap());
    assert_eq!(data[1], CustomDataTypeUInt4Element::try_from(2).unwrap());
    assert_eq!(data[2], CustomDataTypeUInt4Element::try_from(3).unwrap());
    assert_eq!(data[3], CustomDataTypeUInt4Element::try_from(4).unwrap());
    assert_eq!(data[4], CustomDataTypeUInt4Element::try_from(5).unwrap());
    assert_eq!(data[5], CustomDataTypeUInt4Element::try_from(15).unwrap());

    let data: Vec<CustomDataTypeUInt4Element> = array.retrieve_array_subset(&[1..3, 0..1]).unwrap();
    assert_eq!(data[0], CustomDataTypeUInt4Element::try_from(2).unwrap());
    assert_eq!(data[1], CustomDataTypeUInt4Element::try_from(3).unwrap());
}

examples/zarr_v2_to_v3.rs (line 100)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let store = Arc::new(zarrs_storage::store::MemoryStore::new());

    let serde_json::Value::Object(attributes) = serde_json::json!({
        "foo": "bar",
        "baz": 42,
    }) else {
        unreachable!()
    };

    // Create a Zarr V2 group
    let group_metadata: GroupMetadata = GroupMetadataV2::new()
        .with_attributes(attributes.clone())
        .into();
    let group = Group::new_with_metadata(store.clone(), "/group", group_metadata)?;

    // Store the metadata as V2 and V3
    let convert_group_metadata_to_v3 =
        GroupMetadataOptions::default().with_metadata_convert_version(MetadataConvertVersion::V3);
    group.store_metadata()?;
    group.store_metadata_opt(&convert_group_metadata_to_v3)?;
    println!(
        "group/.zgroup (Zarr V2 group metadata):\n{}\n",
        key_to_str(&store, "group/.zgroup")?
    );
    println!(
        "group/.zattrs (Zarr V2 group attributes):\n{}\n",
        key_to_str(&store, "group/.zattrs")?
    );
    println!(
        "group/zarr.json (Zarr V3 equivalent group metadata/attributes):\n{}\n",
        key_to_str(&store, "group/zarr.json")?
    );
    // println!(
    //     "The equivalent Zarr V3 group metadata is\n{}\n",
    //     group.metadata_opt(&convert_group_metadata_to_v3).to_string_pretty()
    // );

    // Create a Zarr V2 array
    let array_metadata = ArrayMetadataV2::new(
        vec![10, 10],
        vec![NonZeroU64::new(5).unwrap(); 2],
        ">f4".into(), // big endian float32
        FillValueMetadata::from(f32::NAN),
        None,
        None,
    )
    .with_dimension_separator(ChunkKeySeparator::Slash)
    .with_order(ArrayMetadataV2Order::F)
    .with_attributes(attributes.clone());
    let array = zarrs::array::Array::new_with_metadata(
        store.clone(),
        "/group/array",
        array_metadata.into(),
    )?;

    // Store the metadata as V2 and V3
    let convert_array_metadata_to_v3 =
        ArrayMetadataOptions::default().with_metadata_convert_version(MetadataConvertVersion::V3);
    array.store_metadata()?;
    array.store_metadata_opt(&convert_array_metadata_to_v3)?;
    println!(
        "group/array/.zarray (Zarr V2 array metadata):\n{}\n",
        key_to_str(&store, "group/array/.zarray")?
    );
    println!(
        "group/array/.zattrs (Zarr V2 array attributes):\n{}\n",
        key_to_str(&store, "group/array/.zattrs")?
    );
    println!(
        "group/array/zarr.json (Zarr V3 equivalent array metadata/attributes):\n{}\n",
        key_to_str(&store, "group/array/zarr.json")?
    );
    // println!(
    //     "The equivalent Zarr V3 array metadata is\n{}\n",
    //     array.metadata_opt(&convert_array_metadata_to_v3).to_string_pretty()
    // );

    array.store_chunk(&[0, 1], &[0.0f32; 5 * 5])?;

    // Print the keys in the store
    println!("The store contains keys:");
    for key in store.list()? {
        println!("  {}", key);
    }

    Ok(())
}

examples/array_write_read_string.rs (lines 75-78)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArrayBytes, data_type};
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![4, 4], // array shape
        vec![2, 2], // regular chunk shape
        data_type::string(),
        "_",
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    array.store_chunk(
        &[0, 0],
        ArrayD::<&str>::from_shape_vec(vec![2, 2], vec!["a", "bb", "ccc", "dddd"]).unwrap(),
    )?;
    array.store_chunk(
        &[0, 1],
        ArrayD::<&str>::from_shape_vec(vec![2, 2], vec!["4444", "333", "22", "1"]).unwrap(),
    )?;
    let subset_all = array.subset_all();
    let data_all: ArrayD<String> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    let ndarray_subset: Array2<&str> = array![["!", "@@"], ["###", "$$$$"]];
    array.store_array_subset(&[1..3, 1..3], ndarray_subset)?;
    let data_all: ArrayD<String> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [1..3, 1..3]:\nndarray::ArrayD<String>\n{data_all}");

    // Retrieve bytes directly, convert into a single string allocation, create a &str ndarray
    // TODO: Add a convenience function for this?
    let data_all: ArrayBytes = array.retrieve_array_subset(&subset_all)?;
    let (bytes, offsets) = data_all.into_variable()?.into_parts();
    let string = String::from_utf8(bytes.into_owned())?;
    let elements = offsets
        .iter()
        .tuple_windows()
        .map(|(&curr, &next)| &string[curr..next])
        .collect::<Vec<&str>>();
    let ndarray = ArrayD::<&str>::from_shape_vec(subset_all.shape_usize(), elements)?;
    println!("ndarray::ArrayD<&str>:\n{ndarray}");

    Ok(())
}

Additional examples can be found in:

• examples/rectangular_array_write_read.rs
• examples/array_write_read.rs
• examples/sharded_array_write_read.rs
• examples/array_write_read_ndarray.rs

pub fn store_chunk_elements<T: Element>( &self, chunk_indices: &[u64], chunk_elements: &[T], ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use store_chunk() instead

Encode chunk_elements and store at chunk_indices.

Use store_chunk_elements_opt to control codec options. A chunk composed entirely of the fill value will not be written to the store.

Errors

Returns an ArrayError if

• the size of T does not match the data type size, or
• a store_chunk error condition is met.

pub fn store_chunk_ndarray<T: Element, D: Dimension>( &self, chunk_indices: &[u64], chunk_array: &ArrayRef<T, D>, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use store_chunk() instead

Available on crate feature ndarray only.

Encode chunk_array and store at chunk_indices.

Use store_chunk_ndarray_opt to control codec options.

Errors

Returns an ArrayError if

• the shape of the array does not match the shape of the chunk,
• a store_chunk_elements error condition is met.

pub fn store_chunks<'a>( &self, chunks: &dyn ArraySubsetTraits, chunks_data: impl IntoArrayBytes<'a>, ) -> Result<(), ArrayError>

Encode chunks_data and store at the chunks with indices represented by the chunks array subset.

Use store_chunks_opt to control codec options. A chunk composed entirely of the fill value will not be written to the store.

Errors

Returns an ArrayError if

• chunks are invalid,
• the length of chunks_data is not equal to the expected length (the product of the number of elements in the chunks and the data type size in bytes),
• there is a codec encoding error, or
• an underlying store error.

Examples found in repository

examples/array_write_read.rs (lines 92-100)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    (0..2).into_par_iter().try_for_each(|i| {
        let chunk_indices: Vec<u64> = vec![0, i];
        let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
            zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
        })?;
        array.store_chunk(
            &chunk_indices,
            vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
        )
    })?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array.store_chunks(
        &[1..2, 0..2],
        &[
            //
            1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            //
            1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
        ],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array.store_array_subset(
        &[3..6, 3..6],
        &[-3.3f32, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array.store_array_subset(
        &[0..8, 6..7],
        &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array.store_chunk_subset(
        // chunk indices
        &[1, 1],
        // subset within chunk
        &[3..4, 0..4],
        &[-7.4f32, -7.5, -7.6, -7.7],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.erase_chunk(&[0, 0])?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.retrieve_chunks(&chunks)?;
    println!("retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.retrieve_array_subset(&subset)?;
    println!("retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

examples/array_write_read_ndarray.rs (line 102)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    (0..2).into_par_iter().try_for_each(|i| {
        let chunk_indices: Vec<u64> = vec![0, i];
        let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
            zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
        })?;
        array.store_chunk(
            &chunk_indices,
            ArrayD::<f32>::from_shape_vec(
                chunk_subset.shape_usize(),
                vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
            )
            .unwrap(),
        )
    })?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    let ndarray_chunks: Array2<f32> = array![
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
    ];
    array.store_chunks(&[1..2, 0..2], ndarray_chunks)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    let ndarray_subset: Array2<f32> =
        array![[-3.3, -3.4, -3.5,], [-4.3, -4.4, -4.5,], [-5.3, -5.4, -5.5],];
    array.store_array_subset(&[3..6, 3..6], ndarray_subset)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    let ndarray_subset: Array2<f32> = array![
        [-0.6],
        [-1.6],
        [-2.6],
        [-3.6],
        [-4.6],
        [-5.6],
        [-6.6],
        [-7.6],
    ];
    array.store_array_subset(&[0..8, 6..7], ndarray_subset)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    let ndarray_chunk_subset: Array2<f32> = array![[-7.4, -7.5, -7.6, -7.7],];
    array.store_chunk_subset(
        // chunk indices
        &[1, 1],
        // subset within chunk
        &[3..4, 0..4],
        ndarray_chunk_subset,
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.erase_chunk(&[0, 0])?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.retrieve_chunks(&chunks)?;
    println!("retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.retrieve_array_subset(&subset)?;
    println!("retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

pub fn store_chunks_elements<T: Element>( &self, chunks: &dyn ArraySubsetTraits, chunks_elements: &[T], ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use store_chunks() instead

Encode chunks_elements and store at the chunks with indices represented by the chunks array subset.

Errors

Returns an ArrayError if

• the size of T does not match the data type size, or
• a store_chunks error condition is met.

pub fn store_chunks_ndarray<T: Element, D: Dimension>( &self, chunks: &dyn ArraySubsetTraits, chunks_array: &ArrayRef<T, D>, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use store_chunks() instead

Available on crate feature ndarray only.

Encode chunks_array and store at the chunks with indices represented by the chunks array subset.

Errors

Returns an ArrayError if

• the shape of the array does not match the shape of the chunks,
• a store_chunks_elements error condition is met.

pub fn erase_metadata(&self) -> Result<(), StorageError>

Erase the metadata with default MetadataEraseVersion options.

Succeeds if the metadata does not exist.

Errors

Returns a StorageError if there is an underlying store error.

pub fn erase_metadata_opt( &self, options: MetadataEraseVersion, ) -> Result<(), StorageError>

Erase the metadata with non-default MetadataEraseVersion options.

Succeeds if the metadata does not exist.

Errors

Returns a StorageError if there is an underlying store error.

pub fn erase_chunk(&self, chunk_indices: &[u64]) -> Result<(), StorageError>

Erase the chunk at chunk_indices.

Succeeds if the chunk does not exist.

Errors

Returns a StorageError if there is an underlying store error.

Examples found in repository

examples/array_write_read.rs (line 132)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    (0..2).into_par_iter().try_for_each(|i| {
        let chunk_indices: Vec<u64> = vec![0, i];
        let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
            zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
        })?;
        array.store_chunk(
            &chunk_indices,
            vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
        )
    })?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array.store_chunks(
        &[1..2, 0..2],
        &[
            //
            1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            //
            1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
        ],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array.store_array_subset(
        &[3..6, 3..6],
        &[-3.3f32, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array.store_array_subset(
        &[0..8, 6..7],
        &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array.store_chunk_subset(
        // chunk indices
        &[1, 1],
        // subset within chunk
        &[3..4, 0..4],
        &[-7.4f32, -7.5, -7.6, -7.7],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.erase_chunk(&[0, 0])?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.retrieve_chunks(&chunks)?;
    println!("retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.retrieve_array_subset(&subset)?;
    println!("retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

examples/array_write_read_ndarray.rs (line 141)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    (0..2).into_par_iter().try_for_each(|i| {
        let chunk_indices: Vec<u64> = vec![0, i];
        let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
            zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
        })?;
        array.store_chunk(
            &chunk_indices,
            ArrayD::<f32>::from_shape_vec(
                chunk_subset.shape_usize(),
                vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
            )
            .unwrap(),
        )
    })?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    let ndarray_chunks: Array2<f32> = array![
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
    ];
    array.store_chunks(&[1..2, 0..2], ndarray_chunks)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    let ndarray_subset: Array2<f32> =
        array![[-3.3, -3.4, -3.5,], [-4.3, -4.4, -4.5,], [-5.3, -5.4, -5.5],];
    array.store_array_subset(&[3..6, 3..6], ndarray_subset)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    let ndarray_subset: Array2<f32> = array![
        [-0.6],
        [-1.6],
        [-2.6],
        [-3.6],
        [-4.6],
        [-5.6],
        [-6.6],
        [-7.6],
    ];
    array.store_array_subset(&[0..8, 6..7], ndarray_subset)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    let ndarray_chunk_subset: Array2<f32> = array![[-7.4, -7.5, -7.6, -7.7],];
    array.store_chunk_subset(
        // chunk indices
        &[1, 1],
        // subset within chunk
        &[3..4, 0..4],
        ndarray_chunk_subset,
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.erase_chunk(&[0, 0])?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.retrieve_chunks(&chunks)?;
    println!("retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.retrieve_array_subset(&subset)?;
    println!("retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

pub fn erase_chunks( &self, chunks: &dyn ArraySubsetTraits, ) -> Result<(), StorageError>

Erase the chunks in chunks.

Errors

Returns a StorageError if there is an underlying store error.

pub fn store_chunk_opt<'a>( &self, chunk_indices: &[u64], chunk_data: impl IntoArrayBytes<'a>, options: &CodecOptions, ) -> Result<(), ArrayError>

Explicit options version of store_chunk.

pub unsafe fn store_encoded_chunk( &self, chunk_indices: &[u64], encoded_chunk_bytes: Bytes, ) -> Result<(), ArrayError>

Store encoded_chunk_bytes at chunk_indices

Safety

The responsibility is on the caller to ensure the chunk is encoded correctly

Errors

Returns StorageError if there is an underlying store error.

pub fn store_chunk_elements_opt<T: Element>( &self, chunk_indices: &[u64], chunk_elements: &[T], options: &CodecOptions, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use store_chunk_opt() instead

Explicit options version of store_chunk_elements.

pub fn store_chunk_ndarray_opt<T: Element, D: Dimension>( &self, chunk_indices: &[u64], chunk_array: &ArrayRef<T, D>, options: &CodecOptions, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use store_chunk_opt() instead

Available on crate feature ndarray only.

Explicit options version of store_chunk_ndarray.

pub fn store_chunks_opt<'a>( &self, chunks: &dyn ArraySubsetTraits, chunks_data: impl IntoArrayBytes<'a>, options: &CodecOptions, ) -> Result<(), ArrayError>

Explicit options version of store_chunks.

pub fn store_chunks_elements_opt<T: Element>( &self, chunks: &dyn ArraySubsetTraits, chunks_elements: &[T], options: &CodecOptions, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use store_chunks_opt() instead

Explicit options version of store_chunks_elements.

pub fn store_chunks_ndarray_opt<T: Element, D: Dimension>( &self, chunks: &dyn ArraySubsetTraits, chunks_array: &ArrayRef<T, D>, options: &CodecOptions, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use store_chunks_opt() instead

Available on crate feature ndarray only.

Explicit options version of store_chunks_ndarray.

impl<TStorage: ?Sized + ReadableWritableStorageTraits + 'static> Array<TStorage>

pub fn readable(&self) -> Array<dyn ReadableStorageTraits>

Return a read-only instantiation of the array.

pub fn store_chunk_subset<'a>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, chunk_subset_data: impl IntoArrayBytes<'a>, ) -> Result<(), ArrayError>

Encode chunk_subset_data and store in chunk_subset of the chunk at chunk_indices with default codec options.

Use store_chunk_subset_opt to control codec options. Prefer to use store_chunk where possible, since this function may decode the chunk before updating it and reencoding it.

Errors

Returns an ArrayError if

• chunk_subset is invalid or out of bounds of the chunk,
• there is a codec encoding error, or
• an underlying store error.

Panics

Panics if attempting to reference a byte beyond usize::MAX.

Examples found in repository

examples/rectangular_array_write_read.rs (lines 126-132)

fn rectangular_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use rayon::prelude::{IntoParallelIterator, ParallelIterator};
    use zarrs::array::{ArraySubset, ZARR_NAN_F32, codec, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        MetadataV3::new_with_configuration(
            "rectangular",
            RectangularChunkGridConfiguration {
                chunk_shape: vec![
                    vec![
                        NonZeroU64::new(1).unwrap(),
                        NonZeroU64::new(2).unwrap(),
                        NonZeroU64::new(3).unwrap(),
                        NonZeroU64::new(2).unwrap(),
                    ]
                    .into(),
                    NonZeroU64::new(4).unwrap().into(),
                ], // chunk sizes
            },
        ),
        data_type::float32(),
        ZARR_NAN_F32,
    )
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(codec::GzipCodec::new(5)?),
    ])
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    // Write some chunks (in parallel)
    (0..4).into_par_iter().try_for_each(|i| {
        let chunk_grid = array.chunk_grid();
        let chunk_indices = vec![i, 0];
        if let Some(chunk_shape) = chunk_grid.chunk_shape(&chunk_indices)? {
            let chunk_array = ndarray::ArrayD::<f32>::from_elem(
                chunk_shape
                    .iter()
                    .map(|u| u.get() as usize)
                    .collect::<Vec<_>>(),
                i as f32,
            );
            array.store_chunk(&chunk_indices, chunk_array)
        } else {
            Err(zarrs::array::ArrayError::InvalidChunkGridIndicesError(
                chunk_indices.to_vec(),
            ))
        }
    })?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write a subset spanning multiple chunks, including updating chunks already written
    array.store_array_subset(
        &[3..6, 3..6], // start
        ndarray::ArrayD::<f32>::from_shape_vec(
            vec![3, 3],
            vec![0.1f32, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9],
        )?,
    )?;

    // Store elements directly, in this case set the 7th column to 123.0
    array.store_array_subset(&[0..8, 6..7], &[123.0f32; 8])?;

    // Store elements directly in a chunk, in this case set the last row of the bottom right chunk
    array.store_chunk_subset(
        // chunk indices
        &[3, 1],
        // subset within chunk
        &[1..2, 0..4],
        &[-4.0f32; 4],
    )?;

    // Read the whole array
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&array.subset_all())?;
    println!("The whole array is:\n{data_all}\n");

    // Read a chunk back from the store
    let chunk_indices = vec![1, 0];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("Chunk [1,0] is:\n{data_chunk}\n");

    // Read the central 4x2 subset of the array
    let subset_4x2 = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_4x2: ArrayD<f32> = array.retrieve_array_subset(&subset_4x2)?;
    println!("The middle 4x2 subset is:\n{data_4x2}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("The Zarr hierarchy tree is:\n{tree}");

    Ok(())
}

examples/array_write_read.rs (lines 121-127)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    (0..2).into_par_iter().try_for_each(|i| {
        let chunk_indices: Vec<u64> = vec![0, i];
        let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
            zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
        })?;
        array.store_chunk(
            &chunk_indices,
            vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
        )
    })?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array.store_chunks(
        &[1..2, 0..2],
        &[
            //
            1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            //
            1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
        ],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array.store_array_subset(
        &[3..6, 3..6],
        &[-3.3f32, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array.store_array_subset(
        &[0..8, 6..7],
        &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array.store_chunk_subset(
        // chunk indices
        &[1, 1],
        // subset within chunk
        &[3..4, 0..4],
        &[-7.4f32, -7.5, -7.6, -7.7],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.erase_chunk(&[0, 0])?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.retrieve_chunks(&chunks)?;
    println!("retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.retrieve_array_subset(&subset)?;
    println!("retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

examples/array_write_read_ndarray.rs (lines 130-136)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    (0..2).into_par_iter().try_for_each(|i| {
        let chunk_indices: Vec<u64> = vec![0, i];
        let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
            zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
        })?;
        array.store_chunk(
            &chunk_indices,
            ArrayD::<f32>::from_shape_vec(
                chunk_subset.shape_usize(),
                vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
            )
            .unwrap(),
        )
    })?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    let ndarray_chunks: Array2<f32> = array![
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
    ];
    array.store_chunks(&[1..2, 0..2], ndarray_chunks)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    let ndarray_subset: Array2<f32> =
        array![[-3.3, -3.4, -3.5,], [-4.3, -4.4, -4.5,], [-5.3, -5.4, -5.5],];
    array.store_array_subset(&[3..6, 3..6], ndarray_subset)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    let ndarray_subset: Array2<f32> = array![
        [-0.6],
        [-1.6],
        [-2.6],
        [-3.6],
        [-4.6],
        [-5.6],
        [-6.6],
        [-7.6],
    ];
    array.store_array_subset(&[0..8, 6..7], ndarray_subset)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    let ndarray_chunk_subset: Array2<f32> = array![[-7.4, -7.5, -7.6, -7.7],];
    array.store_chunk_subset(
        // chunk indices
        &[1, 1],
        // subset within chunk
        &[3..4, 0..4],
        ndarray_chunk_subset,
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.erase_chunk(&[0, 0])?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.retrieve_chunks(&chunks)?;
    println!("retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.retrieve_array_subset(&subset)?;
    println!("retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

pub fn store_chunk_subset_elements<T: Element>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, chunk_subset_elements: &[T], ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use store_chunk_subset() instead

Encode chunk_subset_elements and store in chunk_subset of the chunk at chunk_indices with default codec options.

Use store_chunk_subset_elements_opt to control codec options. Prefer to use store_chunk_elements where possible, since this will decode the chunk before updating it and reencoding it.

Errors

Returns an ArrayError if

• the size of T does not match the data type size, or
• a store_chunk_subset error condition is met.

pub fn store_chunk_subset_ndarray<T: Element, D: Dimension>( &self, chunk_indices: &[u64], chunk_subset_start: &[u64], chunk_subset_array: &ArrayRef<T, D>, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use store_chunk_subset() instead

Available on crate feature ndarray only.

Encode chunk_subset_array and store in chunk_subset of the chunk in the subset starting at chunk_subset_start.

Use store_chunk_subset_ndarray_opt to control codec options. Prefer to use store_chunk_ndarray where possible, since this will decode the chunk before updating it and reencoding it.

Errors

Returns an ArrayError if a store_chunk_subset_elements error condition is met.

pub fn store_array_subset<'a>( &self, array_subset: &dyn ArraySubsetTraits, subset_data: impl IntoArrayBytes<'a>, ) -> Result<(), ArrayError>

Encode subset_data and store in array_subset.

Use store_array_subset_opt to control codec options. Prefer to use store_chunk or store_chunks where possible, since this will decode and encode each chunk intersecting array_subset.

Errors

Returns an ArrayError if

• the dimensionality of array_subset does not match the chunk grid dimensionality
• the length of subset_data does not match the expected length governed by the shape of the array subset and the data type size,
• there is a codec encoding error, or
• an underlying store error.

Examples found in repository

examples/custom_data_type_uint12.rs (line 225)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let fill_value = CustomDataTypeUInt12Element::try_from(15).unwrap();
    let array = ArrayBuilder::new(
        vec![4096, 1], // array shape
        vec![5, 1],    // regular chunk shape
        Arc::new(CustomDataTypeUInt12),
        FillValue::new(fill_value.into_le_bytes().to_vec()),
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .array_to_bytes_codec(Arc::new(zarrs::array::codec::PackBitsCodec::default()))
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data: Vec<CustomDataTypeUInt12Element> = (0..4096)
        .map(|i| CustomDataTypeUInt12Element::try_from(i).unwrap())
        .collect();

    array
        .store_array_subset(&array.subset_all(), &data)
        .unwrap();

    let mut data: Vec<CustomDataTypeUInt12Element> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    for (i, d) in data.drain(0..4096).enumerate() {
        let element = CustomDataTypeUInt12Element::try_from(i as u64).unwrap();
        assert_eq!(d, element);
        let element_pd: Vec<CustomDataTypeUInt12Element> = array
            .retrieve_array_subset(&[(i as u64)..i as u64 + 1, 0..1])
            .unwrap();
        assert_eq!(element_pd[0], element);
    }
}

examples/data_type_optional_nested.rs (line 57)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create an in-memory store
    // let store = Arc::new(zarrs::filesystem::FilesystemStore::new(
    //     "zarrs/tests/data/v3/array_optional_nested.zarr",
    // )?);
    let store = Arc::new(zarrs::storage::store::MemoryStore::new());

    // Build the codec chains for the optional codec
    let array = ArrayBuilder::new(
        vec![4, 4],                                     // 4x4 array
        vec![2, 2],                                     // 2x2 chunks
        data_type::uint8().to_optional().to_optional(), // Optional optional uint8 => Option<Option<u8>>
        FillValue::new_optional_null().into_optional(), // Fill value => Some(None)
    )
    .dimension_names(["y", "x"].into())
    .attributes(
        serde_json::json!({
            "description": r#"A 4x4 array of optional optional uint8 values with some missing data.
The fill value is null on the inner optional layer, i.e. Some(None).
N marks missing (`None`=`null`) values. SN marks `Some(None)`=`[null]` values:
  N  SN   2   3 
  N   5   N   7 
 SN  SN   N   N 
 SN  SN   N   N"#,
        })
        .as_object()
        .unwrap()
        .clone(),
    )
    .build(store.clone(), "/array")?;
    array.store_metadata_opt(
        &zarrs::array::ArrayMetadataOptions::default().with_include_zarrs_metadata(false),
    )?;

    println!("Array metadata:\n{}", array.metadata().to_string_pretty());

    // Create some data with missing values
    let data = ndarray::array![
        [None, Some(None), Some(Some(2u8)), Some(Some(3u8))],
        [None, Some(Some(5u8)), None, Some(Some(7u8))],
        [Some(None), Some(None), None, None],
        [Some(None), Some(None), None, None],
    ]
    .into_dyn();

    // Write the data
    array.store_array_subset(&array.subset_all(), data.clone())?;
    println!("Data written to array.");

    // Read back the data
    let data_read: ArrayD<Option<Option<u8>>> = array.retrieve_array_subset(&array.subset_all())?;

    // Verify data integrity
    assert_eq!(data, data_read);

    // Display the data in a grid format
    println!(
        "Data grid. N marks missing (`None`=`null`) values. SN marks `Some(None)`=`[null]` values"
    );
    println!("    0   1   2   3");
    for y in 0..4 {
        print!("{} ", y);
        for x in 0..4 {
            match data_read[[y, x]] {
                Some(Some(value)) => print!("{:3} ", value),
                Some(None) => print!(" SN "),
                None => print!("  N "),
            }
        }
        println!();
    }
    Ok(())
}

examples/array_write_read_string.rs (line 89)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArrayBytes, data_type};
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![4, 4], // array shape
        vec![2, 2], // regular chunk shape
        data_type::string(),
        "_",
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    array.store_chunk(
        &[0, 0],
        ArrayD::<&str>::from_shape_vec(vec![2, 2], vec!["a", "bb", "ccc", "dddd"]).unwrap(),
    )?;
    array.store_chunk(
        &[0, 1],
        ArrayD::<&str>::from_shape_vec(vec![2, 2], vec!["4444", "333", "22", "1"]).unwrap(),
    )?;
    let subset_all = array.subset_all();
    let data_all: ArrayD<String> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    let ndarray_subset: Array2<&str> = array![["!", "@@"], ["###", "$$$$"]];
    array.store_array_subset(&[1..3, 1..3], ndarray_subset)?;
    let data_all: ArrayD<String> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [1..3, 1..3]:\nndarray::ArrayD<String>\n{data_all}");

    // Retrieve bytes directly, convert into a single string allocation, create a &str ndarray
    // TODO: Add a convenience function for this?
    let data_all: ArrayBytes = array.retrieve_array_subset(&subset_all)?;
    let (bytes, offsets) = data_all.into_variable()?.into_parts();
    let string = String::from_utf8(bytes.into_owned())?;
    let elements = offsets
        .iter()
        .tuple_windows()
        .map(|(&curr, &next)| &string[curr..next])
        .collect::<Vec<&str>>();
    let ndarray = ArrayD::<&str>::from_shape_vec(subset_all.shape_usize(), elements)?;
    println!("ndarray::ArrayD<&str>:\n{ndarray}");

    Ok(())
}

examples/rectangular_array_write_read.rs (lines 114-120)

fn rectangular_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use rayon::prelude::{IntoParallelIterator, ParallelIterator};
    use zarrs::array::{ArraySubset, ZARR_NAN_F32, codec, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        MetadataV3::new_with_configuration(
            "rectangular",
            RectangularChunkGridConfiguration {
                chunk_shape: vec![
                    vec![
                        NonZeroU64::new(1).unwrap(),
                        NonZeroU64::new(2).unwrap(),
                        NonZeroU64::new(3).unwrap(),
                        NonZeroU64::new(2).unwrap(),
                    ]
                    .into(),
                    NonZeroU64::new(4).unwrap().into(),
                ], // chunk sizes
            },
        ),
        data_type::float32(),
        ZARR_NAN_F32,
    )
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(codec::GzipCodec::new(5)?),
    ])
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    // Write some chunks (in parallel)
    (0..4).into_par_iter().try_for_each(|i| {
        let chunk_grid = array.chunk_grid();
        let chunk_indices = vec![i, 0];
        if let Some(chunk_shape) = chunk_grid.chunk_shape(&chunk_indices)? {
            let chunk_array = ndarray::ArrayD::<f32>::from_elem(
                chunk_shape
                    .iter()
                    .map(|u| u.get() as usize)
                    .collect::<Vec<_>>(),
                i as f32,
            );
            array.store_chunk(&chunk_indices, chunk_array)
        } else {
            Err(zarrs::array::ArrayError::InvalidChunkGridIndicesError(
                chunk_indices.to_vec(),
            ))
        }
    })?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write a subset spanning multiple chunks, including updating chunks already written
    array.store_array_subset(
        &[3..6, 3..6], // start
        ndarray::ArrayD::<f32>::from_shape_vec(
            vec![3, 3],
            vec![0.1f32, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9],
        )?,
    )?;

    // Store elements directly, in this case set the 7th column to 123.0
    array.store_array_subset(&[0..8, 6..7], &[123.0f32; 8])?;

    // Store elements directly in a chunk, in this case set the last row of the bottom right chunk
    array.store_chunk_subset(
        // chunk indices
        &[3, 1],
        // subset within chunk
        &[1..2, 0..4],
        &[-4.0f32; 4],
    )?;

    // Read the whole array
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&array.subset_all())?;
    println!("The whole array is:\n{data_all}\n");

    // Read a chunk back from the store
    let chunk_indices = vec![1, 0];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("Chunk [1,0] is:\n{data_chunk}\n");

    // Read the central 4x2 subset of the array
    let subset_4x2 = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_4x2: ArrayD<f32> = array.retrieve_array_subset(&subset_4x2)?;
    println!("The middle 4x2 subset is:\n{data_4x2}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("The Zarr hierarchy tree is:\n{tree}");

    Ok(())
}

examples/array_write_read.rs (lines 105-108)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    (0..2).into_par_iter().try_for_each(|i| {
        let chunk_indices: Vec<u64> = vec![0, i];
        let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
            zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
        })?;
        array.store_chunk(
            &chunk_indices,
            vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
        )
    })?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array.store_chunks(
        &[1..2, 0..2],
        &[
            //
            1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            //
            1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
        ],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array.store_array_subset(
        &[3..6, 3..6],
        &[-3.3f32, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array.store_array_subset(
        &[0..8, 6..7],
        &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array.store_chunk_subset(
        // chunk indices
        &[1, 1],
        // subset within chunk
        &[3..4, 0..4],
        &[-7.4f32, -7.5, -7.6, -7.7],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.erase_chunk(&[0, 0])?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.retrieve_chunks(&chunks)?;
    println!("retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.retrieve_array_subset(&subset)?;
    println!("retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

examples/data_type_optional.rs (line 63)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create an in-memory store
    // let store = Arc::new(zarrs::filesystem::FilesystemStore::new(
    //     "zarrs/tests/data/v3/array_optional.zarr",
    // )?);
    let store = Arc::new(zarrs::storage::store::MemoryStore::new());

    // Build the codec chains for the optional codec
    let array = ArrayBuilder::new(
        vec![4, 4],                       // 4x4 array
        vec![2, 2],                       // 2x2 chunks
        data_type::uint8().to_optional(), // Optional uint8
        FillValue::new_optional_null(),   // Null fill value: [0]
    )
    .dimension_names(["y", "x"].into())
    .attributes(
        serde_json::json!({
            "description": r#"A 4x4 array of optional uint8 values with some missing data.
N marks missing (`None`=`null`) values:
 0  N  2  3 
 N  5  N  7 
 8  9  N  N 
12  N  N  N"#,
        })
        .as_object()
        .unwrap()
        .clone(),
    )
    .build(store.clone(), "/array")?;
    array.store_metadata_opt(
        &zarrs::array::ArrayMetadataOptions::default().with_include_zarrs_metadata(false),
    )?;

    println!("Array metadata:\n{}", array.metadata().to_string_pretty());

    // Create some data with missing values
    let data = ndarray::array![
        [Some(0u8), None, Some(2u8), Some(3u8)],
        [None, Some(5u8), None, Some(7u8)],
        [Some(8u8), Some(9u8), None, None],
        [Some(12u8), None, None, None],
    ]
    .into_dyn();

    // Write the data
    array.store_array_subset(&array.subset_all(), data.clone())?;

    // Read back the data
    let data_read: ArrayD<Option<u8>> = array.retrieve_array_subset(&array.subset_all())?;

    // Verify data integrity
    assert_eq!(data, data_read);

    // Display the data in a grid format
    println!("Data grid, N marks missing (`None`=`null`) values");
    println!("   0  1  2  3");
    for y in 0..4 {
        print!("{} ", y);
        for x in 0..4 {
            match data_read[[y, x]] {
                Some(value) => print!("{:2} ", value),
                None => print!(" N "),
            }
        }
        println!();
    }

    // Print the raw bytes in all chunks
    println!("Raw bytes in all chunks:");
    let chunk_grid_shape = array.chunk_grid_shape();
    for chunk_y in 0..chunk_grid_shape[0] {
        for chunk_x in 0..chunk_grid_shape[1] {
            let chunk_indices = vec![chunk_y, chunk_x];
            let chunk_key = array.chunk_key(&chunk_indices);
            println!("  Chunk [{}, {}] (key: {}):", chunk_y, chunk_x, chunk_key);

            if let Some(chunk_bytes) = store.get(&chunk_key)? {
                println!("    Size: {} bytes", chunk_bytes.len());

                if chunk_bytes.len() >= 16 {
                    // Parse first 8 bytes as mask size (little-endian u64)
                    let mask_size = u64::from_le_bytes([
                        chunk_bytes[0],
                        chunk_bytes[1],
                        chunk_bytes[2],
                        chunk_bytes[3],
                        chunk_bytes[4],
                        chunk_bytes[5],
                        chunk_bytes[6],
                        chunk_bytes[7],
                    ]) as usize;

                    // Parse second 8 bytes as data size (little-endian u64)
                    let data_size = u64::from_le_bytes([
                        chunk_bytes[8],
                        chunk_bytes[9],
                        chunk_bytes[10],
                        chunk_bytes[11],
                        chunk_bytes[12],
                        chunk_bytes[13],
                        chunk_bytes[14],
                        chunk_bytes[15],
                    ]) as usize;

                    // Display mask size header with raw bytes
                    print!("    Mask size: 0b");
                    for byte in &chunk_bytes[0..8] {
                        print!("{:08b}", byte);
                    }
                    println!(" -> {} bytes", mask_size);

                    // Display data size header with raw bytes
                    print!("    Data size: 0b");
                    for byte in &chunk_bytes[8..16] {
                        print!("{:08b}", byte);
                    }
                    println!(" -> {} bytes", data_size);

                    // Show mask and data sections separately
                    if chunk_bytes.len() >= 16 + mask_size + data_size {
                        let mask_start = 16;
                        let data_start = 16 + mask_size;

                        // Show mask as binary
                        if mask_size > 0 {
                            println!("    Mask (binary):");
                            print!("      ");
                            for byte in &chunk_bytes[mask_start..mask_start + mask_size] {
                                print!("0b{:08b} ", byte);
                            }
                            println!();
                        }

                        // Show data as binary
                        if data_size > 0 {
                            println!("    Data (binary):");
                            print!("      ");
                            for byte in &chunk_bytes[data_start..data_start + data_size] {
                                print!("0b{:08b} ", byte);
                            }
                            println!();
                        }
                    }
                } else {
                    panic!("    Chunk too small to parse headers");
                }
            } else {
                println!("    Chunk missing (fill value chunk)");
            }
        }
    }
    Ok(())
}

Additional examples can be found in:

• examples/array_write_read_ndarray.rs

pub fn store_array_subset_elements<T: Element>( &self, array_subset: &dyn ArraySubsetTraits, subset_elements: &[T], ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use store_array_subset() instead

Encode subset_elements and store in array_subset.

Use store_array_subset_elements_opt to control codec options. Prefer to use store_chunk_elements or store_chunks_elements where possible, since this will decode and encode each chunk intersecting array_subset.

Errors

Returns an ArrayError if

• the size of T does not match the data type size, or
• a store_array_subset error condition is met.

pub fn store_array_subset_ndarray<T: Element, D: Dimension>( &self, subset_start: &[u64], subset_array: &ArrayRef<T, D>, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use store_array_subset() instead

Available on crate feature ndarray only.

Encode subset_array and store in the array subset starting at subset_start.

Use store_array_subset_ndarray_opt to control codec options. Prefer to use store_chunk_ndarray or store_chunks_ndarray where possible, since this will decode and encode each chunk intersecting array_subset.

Errors

Returns an ArrayError if a store_array_subset_elements error condition is met.

pub fn compact_chunk( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<bool, ArrayError>

Retrieve the chunk at chunk_indices, compact it if possible, and store the compacted chunk back.

Compaction removes any extraneous data from the encoded chunk representation.

Errors

Returns an ArrayError if

• there is a codec error, or
• an underlying store error.

pub fn store_chunk_subset_opt<'a>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, chunk_subset_data: impl IntoArrayBytes<'a>, options: &CodecOptions, ) -> Result<(), ArrayError>

Explicit options version of store_chunk_subset.

pub fn store_chunk_subset_elements_opt<T: Element>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, chunk_subset_elements: &[T], options: &CodecOptions, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use store_chunk_subset_opt() instead

Explicit options version of store_chunk_subset_elements.

pub fn store_chunk_subset_ndarray_opt<T: Element, D: Dimension>( &self, chunk_indices: &[u64], chunk_subset_start: &[u64], chunk_subset_array: &ArrayRef<T, D>, options: &CodecOptions, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use store_chunk_subset_opt() instead

Available on crate feature ndarray only.

Explicit options version of store_chunk_subset_ndarray.

pub fn store_array_subset_opt<'a>( &self, array_subset: &dyn ArraySubsetTraits, subset_data: impl IntoArrayBytes<'a>, options: &CodecOptions, ) -> Result<(), ArrayError>

Explicit options version of store_array_subset.

pub fn store_array_subset_elements_opt<T: Element>( &self, array_subset: &dyn ArraySubsetTraits, subset_elements: &[T], options: &CodecOptions, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use store_array_subset_opt() instead

Explicit options version of store_array_subset_elements.

pub fn store_array_subset_ndarray_opt<T: Element, D: Dimension>( &self, subset_start: &[u64], subset_array: &ArrayRef<T, D>, options: &CodecOptions, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use store_array_subset_opt() instead

Available on crate feature ndarray only.

Explicit options version of store_array_subset_ndarray.

pub fn partial_encoder( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<Arc<dyn ArrayPartialEncoderTraits>, ArrayError>

Initialises a partial encoder for the chunk at chunk_indices.

Only one partial encoder should be created for a chunk at a time because:

• partial encoders can hold internal state that may become out of sync, and
• parallel writing to the same chunk may result in data loss.

Partial encoding with ArrayPartialEncoderTraits::partial_encode will use parallelism internally where possible.

Errors

Returns an ArrayError if initialisation of the partial encoder fails.

impl<TStorage: ?Sized + AsyncReadableStorageTraits + 'static> Array<TStorage>

pub async fn async_open( storage: Arc<TStorage>, path: &str, ) -> Result<Array<TStorage>, ArrayCreateError>

Available on crate feature async only.

Async variant of open.

Examples found in repository

examples/async_http_array_read.rs (line 50)

async fn http_array_read(backend: Backend) -> Result<(), Box<dyn std::error::Error>> {
    const HTTP_URL: &str =
        "https://raw.githubusercontent.com/zarrs/zarrs/main/zarrs/tests/data/array_write_read.zarr";
    const ARRAY_PATH: &str = "/group/array";

    // Create a HTTP store
    let mut store: AsyncReadableStorage = match backend {
        // Backend::OpenDAL => {
        //     let builder = opendal::services::Http::default().endpoint(HTTP_URL);
        //     let operator = opendal::Operator::new(builder)?.finish();
        //     Arc::new(zarrs_opendal::AsyncOpendalStore::new(operator))
        // }
        Backend::ObjectStore => {
            let options = object_store::ClientOptions::new().with_allow_http(true);
            let store = object_store::http::HttpBuilder::new()
                .with_url(HTTP_URL)
                .with_client_options(options)
                .build()?;
            Arc::new(zarrs_object_store::AsyncObjectStore::new(store))
        }
    };
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Init the existing array, reading metadata
    let array = Array::async_open(store, ARRAY_PATH).await?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Read the whole array
    let data_all: ArrayD<f32> = array
        .async_retrieve_array_subset(&array.subset_all())
        .await?;
    println!("The whole array is:\n{data_all}\n");

    // Read a chunk back from the store
    let chunk_indices = vec![1, 0];
    let data_chunk: ArrayD<f32> = array.async_retrieve_chunk(&chunk_indices).await?;
    println!("Chunk [1,0] is:\n{data_chunk}\n");

    // Read the central 4x2 subset of the array
    let subset_4x2 = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_4x2: ArrayD<f32> = array.async_retrieve_array_subset(&subset_4x2).await?;
    println!("The middle 4x2 subset is:\n{data_4x2}\n");

    Ok(())
}

pub async fn async_open_opt( storage: Arc<TStorage>, path: &str, version: &MetadataRetrieveVersion, ) -> Result<Array<TStorage>, ArrayCreateError>

Available on crate feature async only.

Async variant of open_opt.

pub async fn async_retrieve_chunk_if_exists<T: FromArrayBytes>( &self, chunk_indices: &[u64], ) -> Result<Option<T>, ArrayError>

Available on crate feature async only.

Async variant of retrieve_chunk_if_exists.

pub async fn async_retrieve_chunk_elements_if_exists<T: ElementOwned + MaybeSend + MaybeSync>( &self, chunk_indices: &[u64], ) -> Result<Option<Vec<T>>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_chunk_if_exists instead

Available on crate feature async only.

Async variant of retrieve_chunk_elements_if_exists.

pub async fn async_retrieve_chunk_ndarray_if_exists<T: ElementOwned + MaybeSend + MaybeSync>( &self, chunk_indices: &[u64], ) -> Result<Option<ArrayD<T>>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_chunk_if_exists::<ndarray::ArrayD<T>>() instead

Available on crate features async and ndarray only.

Async variant of retrieve_chunk_ndarray_if_exists.

pub async fn async_retrieve_encoded_chunk( &self, chunk_indices: &[u64], ) -> Result<Option<Bytes>, StorageError>

Available on crate feature async only.

Retrieve the encoded bytes of a chunk.

Errors

Returns a StorageError if there is an underlying store error.

pub async fn async_retrieve_chunk<T: FromArrayBytes>( &self, chunk_indices: &[u64], ) -> Result<T, ArrayError>

Available on crate feature async only.

Async variant of retrieve_chunk.

Examples found in repository

examples/async_http_array_read.rs (line 65)

async fn http_array_read(backend: Backend) -> Result<(), Box<dyn std::error::Error>> {
    const HTTP_URL: &str =
        "https://raw.githubusercontent.com/zarrs/zarrs/main/zarrs/tests/data/array_write_read.zarr";
    const ARRAY_PATH: &str = "/group/array";

    // Create a HTTP store
    let mut store: AsyncReadableStorage = match backend {
        // Backend::OpenDAL => {
        //     let builder = opendal::services::Http::default().endpoint(HTTP_URL);
        //     let operator = opendal::Operator::new(builder)?.finish();
        //     Arc::new(zarrs_opendal::AsyncOpendalStore::new(operator))
        // }
        Backend::ObjectStore => {
            let options = object_store::ClientOptions::new().with_allow_http(true);
            let store = object_store::http::HttpBuilder::new()
                .with_url(HTTP_URL)
                .with_client_options(options)
                .build()?;
            Arc::new(zarrs_object_store::AsyncObjectStore::new(store))
        }
    };
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Init the existing array, reading metadata
    let array = Array::async_open(store, ARRAY_PATH).await?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Read the whole array
    let data_all: ArrayD<f32> = array
        .async_retrieve_array_subset(&array.subset_all())
        .await?;
    println!("The whole array is:\n{data_all}\n");

    // Read a chunk back from the store
    let chunk_indices = vec![1, 0];
    let data_chunk: ArrayD<f32> = array.async_retrieve_chunk(&chunk_indices).await?;
    println!("Chunk [1,0] is:\n{data_chunk}\n");

    // Read the central 4x2 subset of the array
    let subset_4x2 = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_4x2: ArrayD<f32> = array.async_retrieve_array_subset(&subset_4x2).await?;
    println!("The middle 4x2 subset is:\n{data_4x2}\n");

    Ok(())
}

examples/async_array_write_read.rs (line 152)

async fn async_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use futures::StreamExt;
    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;

    // Create a store
    let mut store: AsyncReadableWritableListableStorage = Arc::new(
        zarrs_object_store::AsyncObjectStore::new(object_store::memory::InMemory::new()),
    );
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .async_store_metadata()
        .await?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.async_store_metadata().await?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build_arc(store.clone(), array_path)?;

    // Write array metadata to store
    array.async_store_metadata().await?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    let store_chunk = |i: u64| {
        let array = array.clone();
        async move {
            let chunk_indices: Vec<u64> = vec![0, i];
            let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
                zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
            })?;
            array
                .async_store_chunk(
                    &chunk_indices,
                    vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
                )
                .await
        }
    };
    futures::stream::iter(0..2)
        .map(Ok)
        .try_for_each_concurrent(None, store_chunk)
        .await?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array
        .async_store_chunks(
            &[1..2, 0..2],
            &[
                //
                1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
                //
                1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            ],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array
        .async_store_array_subset(
            &[3..6, 3..6],
            &[-3.3, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array
        .async_store_array_subset(
            &[0..8, 6..7],
            &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array
        .async_store_chunk_subset(
            // chunk indices
            &[1, 1],
            // subset within chunk
            &[3..4, 0..4],
            &[-7.4f32, -7.5, -7.6, -7.7],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.async_erase_chunk(&[0, 0]).await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.async_retrieve_chunk(&chunk_indices).await?;
    println!("async_retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.async_retrieve_chunks(&chunks).await?;
    println!("async_retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.async_retrieve_array_subset(&subset).await?;
    println!("async_retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::async_open(store, "/").await.unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

pub async fn async_retrieve_chunk_elements<T: ElementOwned + MaybeSend + MaybeSync>( &self, chunk_indices: &[u64], ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_chunk::<Vec<T>>() instead

Available on crate feature async only.

Async variant of retrieve_chunk_elements.

pub async fn async_retrieve_chunk_ndarray<T: ElementOwned + MaybeSend + MaybeSync>( &self, chunk_indices: &[u64], ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_chunk::<ndarray::ArrayD<T>>() instead

Available on crate features async and ndarray only.

Async variant of retrieve_chunk_ndarray.

pub async fn async_retrieve_chunks<T: FromArrayBytes>( &self, chunks: &dyn ArraySubsetTraits, ) -> Result<T, ArrayError>

Available on crate feature async only.

Async variant of retrieve_chunks.

Examples found in repository

examples/async_array_write_read.rs (line 157)

async fn async_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use futures::StreamExt;
    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;

    // Create a store
    let mut store: AsyncReadableWritableListableStorage = Arc::new(
        zarrs_object_store::AsyncObjectStore::new(object_store::memory::InMemory::new()),
    );
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .async_store_metadata()
        .await?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.async_store_metadata().await?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build_arc(store.clone(), array_path)?;

    // Write array metadata to store
    array.async_store_metadata().await?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    let store_chunk = |i: u64| {
        let array = array.clone();
        async move {
            let chunk_indices: Vec<u64> = vec![0, i];
            let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
                zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
            })?;
            array
                .async_store_chunk(
                    &chunk_indices,
                    vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
                )
                .await
        }
    };
    futures::stream::iter(0..2)
        .map(Ok)
        .try_for_each_concurrent(None, store_chunk)
        .await?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array
        .async_store_chunks(
            &[1..2, 0..2],
            &[
                //
                1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
                //
                1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            ],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array
        .async_store_array_subset(
            &[3..6, 3..6],
            &[-3.3, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array
        .async_store_array_subset(
            &[0..8, 6..7],
            &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array
        .async_store_chunk_subset(
            // chunk indices
            &[1, 1],
            // subset within chunk
            &[3..4, 0..4],
            &[-7.4f32, -7.5, -7.6, -7.7],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.async_erase_chunk(&[0, 0]).await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.async_retrieve_chunk(&chunk_indices).await?;
    println!("async_retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.async_retrieve_chunks(&chunks).await?;
    println!("async_retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.async_retrieve_array_subset(&subset).await?;
    println!("async_retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::async_open(store, "/").await.unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

pub async fn async_retrieve_chunks_elements<T: ElementOwned + MaybeSend + MaybeSync>( &self, chunks: &dyn ArraySubsetTraits, ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_chunks::<Vec<T>>() instead

Available on crate feature async only.

Async variant of retrieve_chunks_elements.

pub async fn async_retrieve_chunks_ndarray<T: ElementOwned + MaybeSend + MaybeSync>( &self, chunks: &dyn ArraySubsetTraits, ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_chunks::<ndarray::ArrayD<T>>() instead

Available on crate features async and ndarray only.

Async variant of retrieve_chunks_ndarray.

pub async fn async_retrieve_chunk_subset<T: FromArrayBytes>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, ) -> Result<T, ArrayError>

Available on crate feature async only.

Async variant of retrieve_chunk_subset.

pub async fn async_retrieve_chunk_subset_elements<T: ElementOwned + MaybeSend + MaybeSync>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_chunk_subset::<Vec<T>>() instead

Available on crate feature async only.

Async variant of retrieve_chunk_subset_elements.

pub async fn async_retrieve_chunk_subset_ndarray<T: ElementOwned + MaybeSend + MaybeSync>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_chunk_subset::<ndarray::ArrayD<T>>() instead

Available on crate features async and ndarray only.

Async variant of retrieve_chunk_subset_ndarray.

pub async fn async_retrieve_array_subset<T: FromArrayBytes>( &self, array_subset: &dyn ArraySubsetTraits, ) -> Result<T, ArrayError>

Available on crate feature async only.

Async variant of retrieve_array_subset.

Examples found in repository

examples/async_http_array_read.rs (line 59)

async fn http_array_read(backend: Backend) -> Result<(), Box<dyn std::error::Error>> {
    const HTTP_URL: &str =
        "https://raw.githubusercontent.com/zarrs/zarrs/main/zarrs/tests/data/array_write_read.zarr";
    const ARRAY_PATH: &str = "/group/array";

    // Create a HTTP store
    let mut store: AsyncReadableStorage = match backend {
        // Backend::OpenDAL => {
        //     let builder = opendal::services::Http::default().endpoint(HTTP_URL);
        //     let operator = opendal::Operator::new(builder)?.finish();
        //     Arc::new(zarrs_opendal::AsyncOpendalStore::new(operator))
        // }
        Backend::ObjectStore => {
            let options = object_store::ClientOptions::new().with_allow_http(true);
            let store = object_store::http::HttpBuilder::new()
                .with_url(HTTP_URL)
                .with_client_options(options)
                .build()?;
            Arc::new(zarrs_object_store::AsyncObjectStore::new(store))
        }
    };
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Init the existing array, reading metadata
    let array = Array::async_open(store, ARRAY_PATH).await?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Read the whole array
    let data_all: ArrayD<f32> = array
        .async_retrieve_array_subset(&array.subset_all())
        .await?;
    println!("The whole array is:\n{data_all}\n");

    // Read a chunk back from the store
    let chunk_indices = vec![1, 0];
    let data_chunk: ArrayD<f32> = array.async_retrieve_chunk(&chunk_indices).await?;
    println!("Chunk [1,0] is:\n{data_chunk}\n");

    // Read the central 4x2 subset of the array
    let subset_4x2 = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_4x2: ArrayD<f32> = array.async_retrieve_array_subset(&subset_4x2).await?;
    println!("The middle 4x2 subset is:\n{data_4x2}\n");

    Ok(())
}

examples/async_array_write_read.rs (line 94)

async fn async_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use futures::StreamExt;
    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;

    // Create a store
    let mut store: AsyncReadableWritableListableStorage = Arc::new(
        zarrs_object_store::AsyncObjectStore::new(object_store::memory::InMemory::new()),
    );
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .async_store_metadata()
        .await?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.async_store_metadata().await?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build_arc(store.clone(), array_path)?;

    // Write array metadata to store
    array.async_store_metadata().await?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    let store_chunk = |i: u64| {
        let array = array.clone();
        async move {
            let chunk_indices: Vec<u64> = vec![0, i];
            let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
                zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
            })?;
            array
                .async_store_chunk(
                    &chunk_indices,
                    vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
                )
                .await
        }
    };
    futures::stream::iter(0..2)
        .map(Ok)
        .try_for_each_concurrent(None, store_chunk)
        .await?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array
        .async_store_chunks(
            &[1..2, 0..2],
            &[
                //
                1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
                //
                1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            ],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array
        .async_store_array_subset(
            &[3..6, 3..6],
            &[-3.3, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array
        .async_store_array_subset(
            &[0..8, 6..7],
            &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array
        .async_store_chunk_subset(
            // chunk indices
            &[1, 1],
            // subset within chunk
            &[3..4, 0..4],
            &[-7.4f32, -7.5, -7.6, -7.7],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.async_erase_chunk(&[0, 0]).await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.async_retrieve_chunk(&chunk_indices).await?;
    println!("async_retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.async_retrieve_chunks(&chunks).await?;
    println!("async_retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.async_retrieve_array_subset(&subset).await?;
    println!("async_retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::async_open(store, "/").await.unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

pub async fn async_retrieve_array_subset_into( &self, array_subset: &dyn ArraySubsetTraits, output_target: ArrayBytesDecodeIntoTarget<'_>, ) -> Result<(), ArrayError>

Available on crate feature async only.

Async variant of retrieve_array_subset_into.

pub async fn async_retrieve_array_subset_elements<T: ElementOwned + MaybeSend + MaybeSync>( &self, array_subset: &dyn ArraySubsetTraits, ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_array_subset::<Vec<T>>() instead

Available on crate feature async only.

Async variant of retrieve_array_subset_elements.

pub async fn async_retrieve_array_subset_ndarray<T: ElementOwned + MaybeSend + MaybeSync>( &self, array_subset: &dyn ArraySubsetTraits, ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_array_subset::<ndarray::ArrayD<T>>() instead

Available on crate features async and ndarray only.

Async variant of retrieve_array_subset_ndarray.

pub async fn async_partial_decoder( &self, chunk_indices: &[u64], ) -> Result<Arc<dyn AsyncArrayPartialDecoderTraits>, ArrayError>

Available on crate feature async only.

Async variant of partial_decoder.

pub async fn async_retrieve_chunk_if_exists_opt<T: FromArrayBytes>( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<Option<T>, ArrayError>

Available on crate feature async only.

Async variant of retrieve_chunk_if_exists_opt.

pub async fn async_retrieve_chunk_opt<T: FromArrayBytes>( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<T, ArrayError>

Available on crate feature async only.

Async variant of retrieve_chunk_opt.

pub async fn async_retrieve_chunk_elements_if_exists_opt<T: ElementOwned + MaybeSend + MaybeSync>( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<Option<Vec<T>>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_chunk_if_exists_opt::<Vec<T>>() instead

Available on crate feature async only.

Async variant of retrieve_chunk_elements_if_exists_opt.

pub async fn async_retrieve_chunk_elements_opt<T: ElementOwned + MaybeSend + MaybeSync>( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_chunk_opt::<Vec<T>>() instead

Available on crate feature async only.

Async variant of retrieve_chunk_elements_opt.

pub async fn async_retrieve_chunk_ndarray_if_exists_opt<T: ElementOwned + MaybeSend + MaybeSync>( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<Option<ArrayD<T>>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_chunk_if_exists_opt::<ndarray::ArrayD<T>>() instead

Available on crate features async and ndarray only.

Async variant of retrieve_chunk_ndarray_if_exists_opt.

pub async fn async_retrieve_chunk_ndarray_opt<T: ElementOwned + MaybeSend + MaybeSync>( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_chunk_opt::<ndarray::ArrayD<T>>() instead

Available on crate features async and ndarray only.

Async variant of retrieve_chunk_ndarray_opt.

pub async fn async_retrieve_encoded_chunks( &self, chunks: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<Vec<Option<Bytes>>, StorageError>

Available on crate feature async only.

Retrieve the encoded bytes of the chunks in chunks.

The chunks are in order of the chunk indices returned by chunks.indices().into_iter().

Errors

Returns a StorageError if there is an underlying store error.

pub async fn async_retrieve_chunks_opt<T: FromArrayBytes>( &self, chunks: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<T, ArrayError>

Available on crate feature async only.

Async variant of retrieve_chunks_opt.

pub async fn async_retrieve_chunks_elements_opt<T: ElementOwned + MaybeSend + MaybeSync>( &self, chunks: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_chunks_opt::<Vec<T>>() instead

Available on crate feature async only.

Async variant of retrieve_chunks_elements_opt.

pub async fn async_retrieve_chunks_ndarray_opt<T: ElementOwned + MaybeSend + MaybeSync>( &self, chunks: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_chunks_opt::<ndarray::ArrayD<T>>() instead

Available on crate features async and ndarray only.

Async variant of retrieve_chunks_ndarray_opt.

pub async fn async_retrieve_array_subset_opt<T: FromArrayBytes>( &self, array_subset: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<T, ArrayError>

Available on crate feature async only.

Async variant of retrieve_array_subset_opt.

pub async fn async_retrieve_array_subset_into_opt( &self, array_subset: &dyn ArraySubsetTraits, output_target: ArrayBytesDecodeIntoTarget<'_>, options: &CodecOptions, ) -> Result<(), ArrayError>

Available on crate feature async only.

Async variant of retrieve_array_subset_into_opt.

pub async fn async_retrieve_array_subset_elements_opt<T: ElementOwned + MaybeSend + MaybeSync>( &self, array_subset: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_array_subset_opt::<Vec<T>>() instead

Available on crate feature async only.

Async variant of retrieve_array_subset_elements_opt.

pub async fn async_retrieve_array_subset_ndarray_opt<T: ElementOwned + MaybeSend + MaybeSync>( &self, array_subset: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_array_subset_opt::<ndarray::ArrayD<T>>() instead

Available on crate features async and ndarray only.

Async variant of retrieve_array_subset_ndarray_opt.

pub async fn async_retrieve_chunk_subset_opt<T: FromArrayBytes>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<T, ArrayError>

Available on crate feature async only.

Async variant of retrieve_chunk_subset_opt.

pub async fn async_retrieve_chunk_subset_elements_opt<T: ElementOwned + MaybeSend + MaybeSync>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_chunk_subset_opt::<Vec<T>>() instead

Available on crate feature async only.

Async variant of retrieve_chunk_subset_elements_opt.

pub async fn async_retrieve_chunk_subset_ndarray_opt<T: ElementOwned + MaybeSend + MaybeSync>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use async_retrieve_chunk_subset_opt::<ndarray::ArrayD<T>>() instead

Available on crate features async and ndarray only.

Async variant of retrieve_chunk_subset_ndarray_opt.

pub async fn async_partial_decoder_opt( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<Arc<dyn AsyncArrayPartialDecoderTraits>, ArrayError>

Available on crate feature async only.

Async variant of partial_decoder_opt.

impl<TStorage: ?Sized + AsyncWritableStorageTraits + 'static> Array<TStorage>

pub async fn async_store_metadata(&self) -> Result<(), StorageError>

Available on crate feature async only.

Async variant of store_metadata.

Examples found in repository

examples/async_array_write_read.rs (line 65)

async fn async_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use futures::StreamExt;
    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;

    // Create a store
    let mut store: AsyncReadableWritableListableStorage = Arc::new(
        zarrs_object_store::AsyncObjectStore::new(object_store::memory::InMemory::new()),
    );
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .async_store_metadata()
        .await?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.async_store_metadata().await?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build_arc(store.clone(), array_path)?;

    // Write array metadata to store
    array.async_store_metadata().await?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    let store_chunk = |i: u64| {
        let array = array.clone();
        async move {
            let chunk_indices: Vec<u64> = vec![0, i];
            let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
                zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
            })?;
            array
                .async_store_chunk(
                    &chunk_indices,
                    vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
                )
                .await
        }
    };
    futures::stream::iter(0..2)
        .map(Ok)
        .try_for_each_concurrent(None, store_chunk)
        .await?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array
        .async_store_chunks(
            &[1..2, 0..2],
            &[
                //
                1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
                //
                1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            ],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array
        .async_store_array_subset(
            &[3..6, 3..6],
            &[-3.3, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array
        .async_store_array_subset(
            &[0..8, 6..7],
            &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array
        .async_store_chunk_subset(
            // chunk indices
            &[1, 1],
            // subset within chunk
            &[3..4, 0..4],
            &[-7.4f32, -7.5, -7.6, -7.7],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.async_erase_chunk(&[0, 0]).await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.async_retrieve_chunk(&chunk_indices).await?;
    println!("async_retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.async_retrieve_chunks(&chunks).await?;
    println!("async_retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.async_retrieve_array_subset(&subset).await?;
    println!("async_retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::async_open(store, "/").await.unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

pub async fn async_store_metadata_opt( &self, options: &ArrayMetadataOptions, ) -> Result<(), StorageError>

Available on crate feature async only.

Async variant of store_metadata_opt.

pub async fn async_store_chunk<'a>( &self, chunk_indices: &[u64], chunk_data: impl IntoArrayBytes<'a> + MaybeSend, ) -> Result<(), ArrayError>

Available on crate feature async only.

Async variant of store_chunk.

Examples found in repository

examples/async_array_write_read.rs (lines 81-84)

async fn async_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use futures::StreamExt;
    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;

    // Create a store
    let mut store: AsyncReadableWritableListableStorage = Arc::new(
        zarrs_object_store::AsyncObjectStore::new(object_store::memory::InMemory::new()),
    );
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .async_store_metadata()
        .await?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.async_store_metadata().await?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build_arc(store.clone(), array_path)?;

    // Write array metadata to store
    array.async_store_metadata().await?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    let store_chunk = |i: u64| {
        let array = array.clone();
        async move {
            let chunk_indices: Vec<u64> = vec![0, i];
            let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
                zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
            })?;
            array
                .async_store_chunk(
                    &chunk_indices,
                    vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
                )
                .await
        }
    };
    futures::stream::iter(0..2)
        .map(Ok)
        .try_for_each_concurrent(None, store_chunk)
        .await?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array
        .async_store_chunks(
            &[1..2, 0..2],
            &[
                //
                1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
                //
                1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            ],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array
        .async_store_array_subset(
            &[3..6, 3..6],
            &[-3.3, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array
        .async_store_array_subset(
            &[0..8, 6..7],
            &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array
        .async_store_chunk_subset(
            // chunk indices
            &[1, 1],
            // subset within chunk
            &[3..4, 0..4],
            &[-7.4f32, -7.5, -7.6, -7.7],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.async_erase_chunk(&[0, 0]).await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.async_retrieve_chunk(&chunk_indices).await?;
    println!("async_retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.async_retrieve_chunks(&chunks).await?;
    println!("async_retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.async_retrieve_array_subset(&subset).await?;
    println!("async_retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::async_open(store, "/").await.unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

pub async fn async_store_chunk_elements<T: Element + MaybeSend + MaybeSync>( &self, chunk_indices: &[u64], chunk_elements: &[T], ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use async_store_chunk() instead

Available on crate feature async only.

Async variant of store_chunk_elements.

pub async fn async_store_chunk_ndarray<T: Element + MaybeSend + MaybeSync, D: Dimension>( &self, chunk_indices: &[u64], chunk_array: &ArrayRef<T, D>, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use async_store_chunk() instead

Available on crate features async and ndarray only.

Async variant of store_chunk_ndarray.

pub async fn async_store_chunks<'a>( &self, chunks: &dyn ArraySubsetTraits, chunks_data: impl IntoArrayBytes<'a> + MaybeSend, ) -> Result<(), ArrayError>

Available on crate feature async only.

Async variant of store_chunks.

Examples found in repository

examples/async_array_write_read.rs (lines 99-107)

async fn async_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use futures::StreamExt;
    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;

    // Create a store
    let mut store: AsyncReadableWritableListableStorage = Arc::new(
        zarrs_object_store::AsyncObjectStore::new(object_store::memory::InMemory::new()),
    );
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .async_store_metadata()
        .await?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.async_store_metadata().await?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build_arc(store.clone(), array_path)?;

    // Write array metadata to store
    array.async_store_metadata().await?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    let store_chunk = |i: u64| {
        let array = array.clone();
        async move {
            let chunk_indices: Vec<u64> = vec![0, i];
            let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
                zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
            })?;
            array
                .async_store_chunk(
                    &chunk_indices,
                    vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
                )
                .await
        }
    };
    futures::stream::iter(0..2)
        .map(Ok)
        .try_for_each_concurrent(None, store_chunk)
        .await?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array
        .async_store_chunks(
            &[1..2, 0..2],
            &[
                //
                1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
                //
                1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            ],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array
        .async_store_array_subset(
            &[3..6, 3..6],
            &[-3.3, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array
        .async_store_array_subset(
            &[0..8, 6..7],
            &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array
        .async_store_chunk_subset(
            // chunk indices
            &[1, 1],
            // subset within chunk
            &[3..4, 0..4],
            &[-7.4f32, -7.5, -7.6, -7.7],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.async_erase_chunk(&[0, 0]).await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.async_retrieve_chunk(&chunk_indices).await?;
    println!("async_retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.async_retrieve_chunks(&chunks).await?;
    println!("async_retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.async_retrieve_array_subset(&subset).await?;
    println!("async_retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::async_open(store, "/").await.unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

pub async fn async_store_chunks_elements<T: Element + MaybeSend + MaybeSync>( &self, chunks: &dyn ArraySubsetTraits, chunks_elements: &[T], ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use async_store_chunks() instead

Available on crate feature async only.

Async variant of store_chunks_elements.

pub async fn async_store_chunks_ndarray<T: Element + MaybeSend + MaybeSync, D: Dimension>( &self, chunks: &dyn ArraySubsetTraits, chunks_array: &ArrayRef<T, D>, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use async_store_chunks() instead

Available on crate features async and ndarray only.

Async variant of store_chunks_ndarray.

pub async fn async_erase_metadata(&self) -> Result<(), StorageError>

Available on crate feature async only.

Async variant of erase_metadata.

pub async fn async_erase_metadata_opt( &self, options: MetadataEraseVersion, ) -> Result<(), StorageError>

Available on crate feature async only.

Async variant of erase_metadata_opt.

pub async fn async_erase_chunk( &self, chunk_indices: &[u64], ) -> Result<(), StorageError>

Available on crate feature async only.

Async variant of erase_chunk.

Examples found in repository

examples/async_array_write_read.rs (line 146)

async fn async_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use futures::StreamExt;
    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;

    // Create a store
    let mut store: AsyncReadableWritableListableStorage = Arc::new(
        zarrs_object_store::AsyncObjectStore::new(object_store::memory::InMemory::new()),
    );
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .async_store_metadata()
        .await?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.async_store_metadata().await?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build_arc(store.clone(), array_path)?;

    // Write array metadata to store
    array.async_store_metadata().await?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    let store_chunk = |i: u64| {
        let array = array.clone();
        async move {
            let chunk_indices: Vec<u64> = vec![0, i];
            let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
                zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
            })?;
            array
                .async_store_chunk(
                    &chunk_indices,
                    vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
                )
                .await
        }
    };
    futures::stream::iter(0..2)
        .map(Ok)
        .try_for_each_concurrent(None, store_chunk)
        .await?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array
        .async_store_chunks(
            &[1..2, 0..2],
            &[
                //
                1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
                //
                1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            ],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array
        .async_store_array_subset(
            &[3..6, 3..6],
            &[-3.3, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array
        .async_store_array_subset(
            &[0..8, 6..7],
            &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array
        .async_store_chunk_subset(
            // chunk indices
            &[1, 1],
            // subset within chunk
            &[3..4, 0..4],
            &[-7.4f32, -7.5, -7.6, -7.7],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.async_erase_chunk(&[0, 0]).await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.async_retrieve_chunk(&chunk_indices).await?;
    println!("async_retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.async_retrieve_chunks(&chunks).await?;
    println!("async_retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.async_retrieve_array_subset(&subset).await?;
    println!("async_retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::async_open(store, "/").await.unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

pub async fn async_erase_chunks( &self, chunks: &dyn ArraySubsetTraits, ) -> Result<(), StorageError>

Available on crate feature async only.

Async variant of erase_chunks.

pub async fn async_store_chunk_opt<'a>( &self, chunk_indices: &[u64], chunk_data: impl IntoArrayBytes<'a> + MaybeSend, options: &CodecOptions, ) -> Result<(), ArrayError>

Available on crate feature async only.

Async variant of store_chunk_opt.

pub async unsafe fn async_store_encoded_chunk( &self, chunk_indices: &[u64], encoded_chunk_bytes: Bytes, ) -> Result<(), ArrayError>

Available on crate feature async only.

Async variant of store_encoded_chunk

pub async fn async_store_chunk_elements_opt<T: Element + MaybeSend + MaybeSync>( &self, chunk_indices: &[u64], chunk_elements: &[T], options: &CodecOptions, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use async_store_chunk_opt() instead

Available on crate feature async only.

Async variant of store_chunk_elements_opt.

pub async fn async_store_chunk_ndarray_opt<T: Element + MaybeSend + MaybeSync, D: Dimension>( &self, chunk_indices: &[u64], chunk_array: &ArrayRef<T, D>, options: &CodecOptions, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use async_store_chunk_opt() instead

Available on crate features async and ndarray only.

Async variant of store_chunk_ndarray_opt.

pub async fn async_store_chunks_opt<'a>( &self, chunks: &dyn ArraySubsetTraits, chunks_data: impl IntoArrayBytes<'a> + MaybeSend, options: &CodecOptions, ) -> Result<(), ArrayError>

Available on crate feature async only.

Async variant of store_chunks_opt.

pub async fn async_store_chunks_elements_opt<T: Element + MaybeSend + MaybeSync>( &self, chunks: &dyn ArraySubsetTraits, chunks_elements: &[T], options: &CodecOptions, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use async_store_chunks_opt() instead

Available on crate feature async only.

Async variant of store_chunks_elements_opt.

pub async fn async_store_chunks_ndarray_opt<T: Element + MaybeSend + MaybeSync, D: Dimension>( &self, chunks: &dyn ArraySubsetTraits, chunks_array: &ArrayRef<T, D>, options: &CodecOptions, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use async_store_chunks_opt() instead

Available on crate features async and ndarray only.

Async variant of store_chunks_ndarray_opt.

impl<TStorage: ?Sized + AsyncReadableWritableStorageTraits + 'static> Array<TStorage>

pub fn async_readable(&self) -> Array<dyn AsyncReadableStorageTraits>

Available on crate feature async only.

Return a read-only instantiation of the array.

pub async fn async_store_chunk_subset<'a>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, chunk_subset_data: impl IntoArrayBytes<'a> + MaybeSend, ) -> Result<(), ArrayError>

Available on crate feature async only.

Async variant of store_chunk_subset.

Examples found in repository

examples/async_array_write_read.rs (lines 134-140)

async fn async_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use futures::StreamExt;
    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;

    // Create a store
    let mut store: AsyncReadableWritableListableStorage = Arc::new(
        zarrs_object_store::AsyncObjectStore::new(object_store::memory::InMemory::new()),
    );
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .async_store_metadata()
        .await?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.async_store_metadata().await?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build_arc(store.clone(), array_path)?;

    // Write array metadata to store
    array.async_store_metadata().await?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    let store_chunk = |i: u64| {
        let array = array.clone();
        async move {
            let chunk_indices: Vec<u64> = vec![0, i];
            let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
                zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
            })?;
            array
                .async_store_chunk(
                    &chunk_indices,
                    vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
                )
                .await
        }
    };
    futures::stream::iter(0..2)
        .map(Ok)
        .try_for_each_concurrent(None, store_chunk)
        .await?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array
        .async_store_chunks(
            &[1..2, 0..2],
            &[
                //
                1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
                //
                1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            ],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array
        .async_store_array_subset(
            &[3..6, 3..6],
            &[-3.3, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array
        .async_store_array_subset(
            &[0..8, 6..7],
            &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array
        .async_store_chunk_subset(
            // chunk indices
            &[1, 1],
            // subset within chunk
            &[3..4, 0..4],
            &[-7.4f32, -7.5, -7.6, -7.7],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.async_erase_chunk(&[0, 0]).await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.async_retrieve_chunk(&chunk_indices).await?;
    println!("async_retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.async_retrieve_chunks(&chunks).await?;
    println!("async_retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.async_retrieve_array_subset(&subset).await?;
    println!("async_retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::async_open(store, "/").await.unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

pub async fn async_store_chunk_subset_elements<T: Element + MaybeSend + MaybeSync>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, chunk_subset_elements: &[T], ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use async_store_chunk_subset() instead

Available on crate feature async only.

Async variant of store_chunk_subset_elements.

pub async fn async_store_chunk_subset_ndarray<T: Element + MaybeSend + MaybeSync, D: Dimension>( &self, chunk_indices: &[u64], chunk_subset_start: &[u64], chunk_subset_array: &ArrayRef<T, D>, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use async_store_chunk_subset() instead

Available on crate features async and ndarray only.

Async variant of store_chunk_subset_ndarray.

pub async fn async_store_array_subset<'a>( &self, array_subset: &dyn ArraySubsetTraits, subset_data: impl IntoArrayBytes<'a> + MaybeSend, ) -> Result<(), ArrayError>

Available on crate feature async only.

Async variant of store_array_subset.

Examples found in repository

examples/async_array_write_read.rs (lines 114-117)

async fn async_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use futures::StreamExt;
    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;

    // Create a store
    let mut store: AsyncReadableWritableListableStorage = Arc::new(
        zarrs_object_store::AsyncObjectStore::new(object_store::memory::InMemory::new()),
    );
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .async_store_metadata()
        .await?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.async_store_metadata().await?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build_arc(store.clone(), array_path)?;

    // Write array metadata to store
    array.async_store_metadata().await?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    let store_chunk = |i: u64| {
        let array = array.clone();
        async move {
            let chunk_indices: Vec<u64> = vec![0, i];
            let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
                zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
            })?;
            array
                .async_store_chunk(
                    &chunk_indices,
                    vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
                )
                .await
        }
    };
    futures::stream::iter(0..2)
        .map(Ok)
        .try_for_each_concurrent(None, store_chunk)
        .await?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array
        .async_store_chunks(
            &[1..2, 0..2],
            &[
                //
                1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
                //
                1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            ],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array
        .async_store_array_subset(
            &[3..6, 3..6],
            &[-3.3, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array
        .async_store_array_subset(
            &[0..8, 6..7],
            &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array
        .async_store_chunk_subset(
            // chunk indices
            &[1, 1],
            // subset within chunk
            &[3..4, 0..4],
            &[-7.4f32, -7.5, -7.6, -7.7],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.async_erase_chunk(&[0, 0]).await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.async_retrieve_chunk(&chunk_indices).await?;
    println!("async_retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.async_retrieve_chunks(&chunks).await?;
    println!("async_retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.async_retrieve_array_subset(&subset).await?;
    println!("async_retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::async_open(store, "/").await.unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

pub async fn async_store_array_subset_elements<T: Element + MaybeSend + MaybeSync>( &self, array_subset: &dyn ArraySubsetTraits, subset_elements: &[T], ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use async_store_array_subset() instead

Available on crate feature async only.

Async variant of store_array_subset_elements.

pub async fn async_store_array_subset_ndarray<T: Element + MaybeSend + MaybeSync, D: Dimension>( &self, subset_start: &[u64], subset_array: &ArrayRef<T, D>, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use async_store_array_subset() instead

Available on crate features async and ndarray only.

Async variant of store_array_subset_ndarray.

pub async fn async_compact_chunk( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<bool, ArrayError>

Available on crate feature async only.

Async variant of compact_chunk.

pub async fn async_store_chunk_subset_opt<'a>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, chunk_subset_data: impl IntoArrayBytes<'a> + MaybeSend, options: &CodecOptions, ) -> Result<(), ArrayError>

Available on crate feature async only.

Async variant of store_chunk_subset_opt.

pub async fn async_store_chunk_subset_elements_opt<T: Element + MaybeSend + MaybeSync>( &self, chunk_indices: &[u64], chunk_subset: &dyn ArraySubsetTraits, chunk_subset_elements: &[T], options: &CodecOptions, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use async_store_chunk_subset_opt() instead

Available on crate feature async only.

Async variant of store_chunk_subset_elements_opt.

pub async fn async_store_chunk_subset_ndarray_opt<T: Element + MaybeSend + MaybeSync, D: Dimension>( &self, chunk_indices: &[u64], chunk_subset_start: &[u64], chunk_subset_array: &ArrayRef<T, D>, options: &CodecOptions, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use async_store_chunk_subset_opt() instead

Available on crate feature async only.

Async variant of store_chunk_subset_ndarray_opt.

pub async fn async_store_array_subset_opt<'a>( &self, array_subset: &dyn ArraySubsetTraits, subset_data: impl IntoArrayBytes<'a> + MaybeSend, options: &CodecOptions, ) -> Result<(), ArrayError>

Available on crate feature async only.

Async variant of store_array_subset_opt.

pub async fn async_store_array_subset_elements_opt<T: Element + MaybeSend + MaybeSync>( &self, array_subset: &dyn ArraySubsetTraits, subset_elements: &[T], options: &CodecOptions, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use async_store_array_subset_opt() instead

Available on crate feature async only.

Async variant of store_array_subset_elements_opt.

pub async fn async_store_array_subset_ndarray_opt<T: Element + MaybeSend + MaybeSync, D: Dimension>( &self, subset_start: &[u64], subset_array: &ArrayRef<T, D>, options: &CodecOptions, ) -> Result<(), ArrayError>

👎Deprecated since 0.23.0: Use async_store_array_subset_opt() instead

Available on crate features async and ndarray only.

Async variant of store_array_subset_ndarray_opt.

pub async fn async_partial_encoder( &self, chunk_indices: &[u64], options: &CodecOptions, ) -> Result<Arc<dyn AsyncArrayPartialEncoderTraits>, ArrayError>

Available on crate feature async only.

Initialises an asynchronous partial encoder for the chunk at chunk_indices.

Only one partial encoder should be created for a chunk at a time because:

• partial encoders can hold internal state that may become out of sync, and
• parallel writing to the same chunk may result in data loss.

Partial encoding with AsyncArrayPartialEncoderTraits::partial_encode will use parallelism internally where possible.

Errors

Returns an ArrayError if initialisation of the partial encoder fails.

impl<TStorage: ?Sized> Array<TStorage>

pub fn with_storage<TStorage2: ?Sized>( &self, storage: Arc<TStorage2>, ) -> Array<TStorage2>

Replace the storage backing an array.

pub fn new_with_metadata( storage: Arc<TStorage>, path: &str, metadata: ArrayMetadata, ) -> Result<Self, ArrayCreateError>

Create an array in storage at path with metadata. This does not write to the store, use store_metadata to write metadata to storage.

Errors

Returns ArrayCreateError if:

• any metadata is invalid or,
• a plugin (e.g. data type/chunk grid/chunk key encoding/codec/storage transformer) is invalid.

Examples found in repository

examples/zarr_v2_to_v3.rs (lines 72-76)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let store = Arc::new(zarrs_storage::store::MemoryStore::new());

    let serde_json::Value::Object(attributes) = serde_json::json!({
        "foo": "bar",
        "baz": 42,
    }) else {
        unreachable!()
    };

    // Create a Zarr V2 group
    let group_metadata: GroupMetadata = GroupMetadataV2::new()
        .with_attributes(attributes.clone())
        .into();
    let group = Group::new_with_metadata(store.clone(), "/group", group_metadata)?;

    // Store the metadata as V2 and V3
    let convert_group_metadata_to_v3 =
        GroupMetadataOptions::default().with_metadata_convert_version(MetadataConvertVersion::V3);
    group.store_metadata()?;
    group.store_metadata_opt(&convert_group_metadata_to_v3)?;
    println!(
        "group/.zgroup (Zarr V2 group metadata):\n{}\n",
        key_to_str(&store, "group/.zgroup")?
    );
    println!(
        "group/.zattrs (Zarr V2 group attributes):\n{}\n",
        key_to_str(&store, "group/.zattrs")?
    );
    println!(
        "group/zarr.json (Zarr V3 equivalent group metadata/attributes):\n{}\n",
        key_to_str(&store, "group/zarr.json")?
    );
    // println!(
    //     "The equivalent Zarr V3 group metadata is\n{}\n",
    //     group.metadata_opt(&convert_group_metadata_to_v3).to_string_pretty()
    // );

    // Create a Zarr V2 array
    let array_metadata = ArrayMetadataV2::new(
        vec![10, 10],
        vec![NonZeroU64::new(5).unwrap(); 2],
        ">f4".into(), // big endian float32
        FillValueMetadata::from(f32::NAN),
        None,
        None,
    )
    .with_dimension_separator(ChunkKeySeparator::Slash)
    .with_order(ArrayMetadataV2Order::F)
    .with_attributes(attributes.clone());
    let array = zarrs::array::Array::new_with_metadata(
        store.clone(),
        "/group/array",
        array_metadata.into(),
    )?;

    // Store the metadata as V2 and V3
    let convert_array_metadata_to_v3 =
        ArrayMetadataOptions::default().with_metadata_convert_version(MetadataConvertVersion::V3);
    array.store_metadata()?;
    array.store_metadata_opt(&convert_array_metadata_to_v3)?;
    println!(
        "group/array/.zarray (Zarr V2 array metadata):\n{}\n",
        key_to_str(&store, "group/array/.zarray")?
    );
    println!(
        "group/array/.zattrs (Zarr V2 array attributes):\n{}\n",
        key_to_str(&store, "group/array/.zattrs")?
    );
    println!(
        "group/array/zarr.json (Zarr V3 equivalent array metadata/attributes):\n{}\n",
        key_to_str(&store, "group/array/zarr.json")?
    );
    // println!(
    //     "The equivalent Zarr V3 array metadata is\n{}\n",
    //     array.metadata_opt(&convert_array_metadata_to_v3).to_string_pretty()
    // );

    array.store_chunk(&[0, 1], &[0.0f32; 5 * 5])?;

    // Print the keys in the store
    println!("The store contains keys:");
    for key in store.list()? {
        println!("  {}", key);
    }

    Ok(())
}

pub fn with_codec_options(self, codec_options: CodecOptions) -> Self

Set the codec options.

pub fn set_codec_options(&mut self, codec_options: CodecOptions) -> &mut Self

Set the codec options.

pub fn with_metadata_options( self, metadata_options: ArrayMetadataOptions, ) -> Self

Set the metadata options.

pub fn set_metadata_options( &mut self, metadata_options: ArrayMetadataOptions, ) -> &mut Self

Set the metadata options.

pub fn storage(&self) -> Arc<TStorage>

Get the underlying storage backing the array.

pub const fn path(&self) -> &NodePath

Get the node path.

pub const fn data_type(&self) -> &DataType

Get the data type.

pub const fn fill_value(&self) -> &FillValue

Get the fill value.

pub fn shape(&self) -> &[u64]

Get the array shape.

pub fn set_shape( &mut self, array_shape: ArrayShape, ) -> Result<&mut Self, ArrayCreateError>

Set the array shape.

Errors

Returns an ArrayCreateError if the chunk grid is not compatible with array_shape.

pub unsafe fn set_shape_and_chunk_grid( &mut self, array_shape: ArrayShape, chunk_grid_metadata: impl Into<ArrayBuilderChunkGridMetadata>, ) -> Result<&mut Self, ArrayCreateError>

Set the array shape and chunk grid from chunk grid metadata.

This method allows setting both the array shape and chunk grid simultaneously. Some chunk grids depend on the array shape (e.g. rectilinear), so this method ensures that the chunk grid is correctly configured for the new array shape.

Errors

Returns an ArrayCreateError if:

• the chunk grid is not compatible with array_shape, or
• the chunk grid metadata is invalid.

Safety

This method does not validate that existing chunks in the store are compatible with the new chunk grid. If the chunk grid is changed such that existing chunks are no longer valid, subsequent read or write operations may fail or produce incorrect results.

It is the caller’s responsibility to ensure that the new chunk grid is compatible with any existing data in the store. This may involve deleting or rewriting existing chunks to match the new chunk grid. Use with caution!

pub fn dimensionality(&self) -> usize

Get the array dimensionality.

pub fn codecs(&self) -> Arc<CodecChain>

Get the codecs.

pub const fn chunk_grid(&self) -> &ChunkGrid

Get the chunk grid.

Examples found in repository

examples/rectangular_array_write_read.rs (line 90)

fn rectangular_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use rayon::prelude::{IntoParallelIterator, ParallelIterator};
    use zarrs::array::{ArraySubset, ZARR_NAN_F32, codec, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        MetadataV3::new_with_configuration(
            "rectangular",
            RectangularChunkGridConfiguration {
                chunk_shape: vec![
                    vec![
                        NonZeroU64::new(1).unwrap(),
                        NonZeroU64::new(2).unwrap(),
                        NonZeroU64::new(3).unwrap(),
                        NonZeroU64::new(2).unwrap(),
                    ]
                    .into(),
                    NonZeroU64::new(4).unwrap().into(),
                ], // chunk sizes
            },
        ),
        data_type::float32(),
        ZARR_NAN_F32,
    )
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(codec::GzipCodec::new(5)?),
    ])
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    // Write some chunks (in parallel)
    (0..4).into_par_iter().try_for_each(|i| {
        let chunk_grid = array.chunk_grid();
        let chunk_indices = vec![i, 0];
        if let Some(chunk_shape) = chunk_grid.chunk_shape(&chunk_indices)? {
            let chunk_array = ndarray::ArrayD::<f32>::from_elem(
                chunk_shape
                    .iter()
                    .map(|u| u.get() as usize)
                    .collect::<Vec<_>>(),
                i as f32,
            );
            array.store_chunk(&chunk_indices, chunk_array)
        } else {
            Err(zarrs::array::ArrayError::InvalidChunkGridIndicesError(
                chunk_indices.to_vec(),
            ))
        }
    })?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write a subset spanning multiple chunks, including updating chunks already written
    array.store_array_subset(
        &[3..6, 3..6], // start
        ndarray::ArrayD::<f32>::from_shape_vec(
            vec![3, 3],
            vec![0.1f32, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9],
        )?,
    )?;

    // Store elements directly, in this case set the 7th column to 123.0
    array.store_array_subset(&[0..8, 6..7], &[123.0f32; 8])?;

    // Store elements directly in a chunk, in this case set the last row of the bottom right chunk
    array.store_chunk_subset(
        // chunk indices
        &[3, 1],
        // subset within chunk
        &[1..2, 0..4],
        &[-4.0f32; 4],
    )?;

    // Read the whole array
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&array.subset_all())?;
    println!("The whole array is:\n{data_all}\n");

    // Read a chunk back from the store
    let chunk_indices = vec![1, 0];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("Chunk [1,0] is:\n{data_chunk}\n");

    // Read the central 4x2 subset of the array
    let subset_4x2 = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_4x2: ArrayD<f32> = array.retrieve_array_subset(&subset_4x2)?;
    println!("The middle 4x2 subset is:\n{data_4x2}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("The Zarr hierarchy tree is:\n{tree}");

    Ok(())
}

examples/array_write_read.rs (line 78)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    (0..2).into_par_iter().try_for_each(|i| {
        let chunk_indices: Vec<u64> = vec![0, i];
        let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
            zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
        })?;
        array.store_chunk(
            &chunk_indices,
            vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
        )
    })?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array.store_chunks(
        &[1..2, 0..2],
        &[
            //
            1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            //
            1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
        ],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array.store_array_subset(
        &[3..6, 3..6],
        &[-3.3f32, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array.store_array_subset(
        &[0..8, 6..7],
        &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array.store_chunk_subset(
        // chunk indices
        &[1, 1],
        // subset within chunk
        &[3..4, 0..4],
        &[-7.4f32, -7.5, -7.6, -7.7],
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.erase_chunk(&[0, 0])?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.retrieve_chunks(&chunks)?;
    println!("retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.retrieve_array_subset(&subset)?;
    println!("retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

examples/sharded_array_write_read.rs (line 84)

fn sharded_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use rayon::prelude::{IntoParallelIterator, ParallelIterator};
    use zarrs::array::{ArraySubset, codec, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/sharded_array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    // Create an array
    let array_path = "/group/array";
    let subchunk_shape = vec![4, 4];
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 8], // chunk (shard) shape
        data_type::uint16(),
        0u16,
    )
    .subchunk_shape(subchunk_shape.clone())
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(codec::GzipCodec::new(5)?),
    ])
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    // The array metadata is
    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Use default codec options (concurrency etc)
    let options = CodecOptions::default();

    // Write some shards (in parallel)
    (0..2).into_par_iter().try_for_each(|s| {
        let chunk_grid = array.chunk_grid();
        let chunk_indices = vec![s, 0];
        if let Some(chunk_shape) = chunk_grid.chunk_shape(&chunk_indices)? {
            let chunk_array = ndarray::ArrayD::<u16>::from_shape_fn(
                chunk_shape
                    .iter()
                    .map(|u| u.get() as usize)
                    .collect::<Vec<_>>(),
                |ij| {
                    (s * chunk_shape[0].get() * chunk_shape[1].get()
                        + ij[0] as u64 * chunk_shape[1].get()
                        + ij[1] as u64) as u16
                },
            );
            array.store_chunk(&chunk_indices, chunk_array)
        } else {
            Err(zarrs::array::ArrayError::InvalidChunkGridIndicesError(
                chunk_indices.to_vec(),
            ))
        }
    })?;

    // Read the whole array
    let data_all: ArrayD<u16> = array.retrieve_array_subset(&array.subset_all())?;
    println!("The whole array is:\n{data_all}\n");

    // Read a shard back from the store
    let shard_indices = vec![1, 0];
    let data_shard: ArrayD<u16> = array.retrieve_chunk(&shard_indices)?;
    println!("Shard [1,0] is:\n{data_shard}\n");

    // Read a subchunk from the store
    let subset_chunk_1_0 = ArraySubset::new_with_ranges(&[4..8, 0..4]);
    let data_chunk: ArrayD<u16> = array.retrieve_array_subset(&subset_chunk_1_0)?;
    println!("Chunk [1,0] is:\n{data_chunk}\n");

    // Read the central 4x2 subset of the array
    let subset_4x2 = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_4x2: ArrayD<u16> = array.retrieve_array_subset(&subset_4x2)?;
    println!("The middle 4x2 subset is:\n{data_4x2}\n");

    // Decode subchunks
    // In some cases, it might be preferable to decode subchunks in a shard directly.
    // If using the partial decoder, then the shard index will only be read once from the store.
    let partial_decoder = array.partial_decoder(&[0, 0])?;
    println!("Decoded subchunks:");
    for subchunk_subset in [
        ArraySubset::new_with_start_shape(vec![0, 0], subchunk_shape.clone())?,
        ArraySubset::new_with_start_shape(vec![0, 4], subchunk_shape.clone())?,
    ] {
        println!("{subchunk_subset}");
        let decoded_subchunk_bytes = partial_decoder.partial_decode(&subchunk_subset, &options)?;
        let ndarray = bytes_to_ndarray::<u16>(
            &subchunk_shape,
            decoded_subchunk_bytes.into_fixed()?.into_owned(),
        )?;
        println!("{ndarray}\n");
    }

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("The Zarr hierarchy tree is:\n{}", tree);

    println!(
        "The keys in the store are:\n[{}]",
        store.list().unwrap_or_default().iter().format(", ")
    );

    Ok(())
}

examples/array_write_read_ndarray.rs (line 78)

fn array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;
    use zarrs::storage::store;

    // Create a store
    // let path = tempfile::TempDir::new()?;
    // let mut store: ReadableWritableListableStorage =
    //     Arc::new(zarrs::filesystem::FilesystemStore::new(path.path())?);
    // let mut store: ReadableWritableListableStorage = Arc::new(
    //     zarrs::filesystem::FilesystemStore::new("zarrs/tests/data/array_write_read.zarr")?,
    // );
    let mut store: ReadableWritableListableStorage = Arc::new(store::MemoryStore::new());
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .store_metadata()?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.store_metadata()?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build(store.clone(), array_path)?;

    // Write array metadata to store
    array.store_metadata()?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    (0..2).into_par_iter().try_for_each(|i| {
        let chunk_indices: Vec<u64> = vec![0, i];
        let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
            zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
        })?;
        array.store_chunk(
            &chunk_indices,
            ArrayD::<f32>::from_shape_vec(
                chunk_subset.shape_usize(),
                vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
            )
            .unwrap(),
        )
    })?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    let ndarray_chunks: Array2<f32> = array![
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
        [1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,],
    ];
    array.store_chunks(&[1..2, 0..2], ndarray_chunks)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    let ndarray_subset: Array2<f32> =
        array![[-3.3, -3.4, -3.5,], [-4.3, -4.4, -4.5,], [-5.3, -5.4, -5.5],];
    array.store_array_subset(&[3..6, 3..6], ndarray_subset)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    let ndarray_subset: Array2<f32> = array![
        [-0.6],
        [-1.6],
        [-2.6],
        [-3.6],
        [-4.6],
        [-5.6],
        [-6.6],
        [-7.6],
    ];
    array.store_array_subset(&[0..8, 6..7], ndarray_subset)?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    let ndarray_chunk_subset: Array2<f32> = array![[-7.4, -7.5, -7.6, -7.7],];
    array.store_chunk_subset(
        // chunk indices
        &[1, 1],
        // subset within chunk
        &[3..4, 0..4],
        ndarray_chunk_subset,
    )?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.erase_chunk(&[0, 0])?;
    let data_all: ArrayD<f32> = array.retrieve_array_subset(&subset_all)?;
    println!("erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.retrieve_chunk(&chunk_indices)?;
    println!("retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.retrieve_chunks(&chunks)?;
    println!("retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.retrieve_array_subset(&subset)?;
    println!("retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::open(&store, "/").unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

examples/async_array_write_read.rs (line 77)

async fn async_array_write_read() -> Result<(), Box<dyn std::error::Error>> {
    use std::sync::Arc;

    use futures::StreamExt;
    use zarrs::array::{ArraySubset, ZARR_NAN_F32, data_type};
    use zarrs::node::Node;

    // Create a store
    let mut store: AsyncReadableWritableListableStorage = Arc::new(
        zarrs_object_store::AsyncObjectStore::new(object_store::memory::InMemory::new()),
    );
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Create the root group
    zarrs::group::GroupBuilder::new()
        .build(store.clone(), "/")?
        .async_store_metadata()
        .await?;

    // Create a group with attributes
    let group_path = "/group";
    let mut group = zarrs::group::GroupBuilder::new().build(store.clone(), group_path)?;
    group
        .attributes_mut()
        .insert("foo".into(), serde_json::Value::String("bar".into()));
    group.async_store_metadata().await?;

    println!(
        "The group metadata is:\n{}\n",
        group.metadata().to_string_pretty()
    );

    // Create an array
    let array_path = "/group/array";
    let array = zarrs::array::ArrayBuilder::new(
        vec![8, 8], // array shape
        vec![4, 4], // regular chunk shape
        data_type::float32(),
        ZARR_NAN_F32,
    )
    // .bytes_to_bytes_codecs(vec![]) // uncompressed
    .dimension_names(["y", "x"].into())
    // .storage_transformers(vec![].into())
    .build_arc(store.clone(), array_path)?;

    // Write array metadata to store
    array.async_store_metadata().await?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Write some chunks
    let store_chunk = |i: u64| {
        let array = array.clone();
        async move {
            let chunk_indices: Vec<u64> = vec![0, i];
            let chunk_subset = array.chunk_grid().subset(&chunk_indices)?.ok_or_else(|| {
                zarrs::array::ArrayError::InvalidChunkGridIndicesError(chunk_indices.to_vec())
            })?;
            array
                .async_store_chunk(
                    &chunk_indices,
                    vec![i as f32 * 0.1; chunk_subset.num_elements() as usize],
                )
                .await
        }
    };
    futures::stream::iter(0..2)
        .map(Ok)
        .try_for_each_concurrent(None, store_chunk)
        .await?;

    let subset_all = array.subset_all();
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk [0, 0] and [0, 1]:\n{data_all:+4.1}\n");

    // Store multiple chunks
    array
        .async_store_chunks(
            &[1..2, 0..2],
            &[
                //
                1.0f32, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
                //
                1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1, 1.0, 1.0, 1.0, 1.0, 1.1, 1.1, 1.1, 1.1,
            ],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunks [1..2, 0..2]:\n{data_all:+4.1}\n");

    // Write a subset spanning multiple chunks, including updating chunks already written
    array
        .async_store_array_subset(
            &[3..6, 3..6],
            &[-3.3, -3.4, -3.5, -4.3, -4.4, -4.5, -5.3, -5.4, -5.5],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [3..6, 3..6]:\n{data_all:+4.1}\n");

    // Store array subset
    array
        .async_store_array_subset(
            &[0..8, 6..7],
            &[-0.6f32, -1.6, -2.6, -3.6, -4.6, -5.6, -6.6, -7.6],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_array_subset [0..8, 6..7]:\n{data_all:+4.1}\n");

    // Store chunk subset
    array
        .async_store_chunk_subset(
            // chunk indices
            &[1, 1],
            // subset within chunk
            &[3..4, 0..4],
            &[-7.4f32, -7.5, -7.6, -7.7],
        )
        .await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_store_chunk_subset [3..4, 0..4] of chunk [1, 1]:\n{data_all:+4.1}\n");

    // Erase a chunk
    array.async_erase_chunk(&[0, 0]).await?;
    let data_all: ArrayD<f32> = array.async_retrieve_array_subset(&subset_all).await?;
    println!("async_erase_chunk [0, 0]:\n{data_all:+4.1}\n");

    // Read a chunk
    let chunk_indices = vec![0, 1];
    let data_chunk: ArrayD<f32> = array.async_retrieve_chunk(&chunk_indices).await?;
    println!("async_retrieve_chunk [0, 1]:\n{data_chunk:+4.1}\n");

    // Read chunks
    let chunks = ArraySubset::new_with_ranges(&[0..2, 1..2]);
    let data_chunks: ArrayD<f32> = array.async_retrieve_chunks(&chunks).await?;
    println!("async_retrieve_chunks [0..2, 1..2]:\n{data_chunks:+4.1}\n");

    // Retrieve an array subset
    let subset = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_subset: ArrayD<f32> = array.async_retrieve_array_subset(&subset).await?;
    println!("async_retrieve_array_subset [2..6, 3..5]:\n{data_subset:+4.1}\n");

    // Show the hierarchy
    let node = Node::async_open(store, "/").await.unwrap();
    let tree = node.hierarchy_tree();
    println!("hierarchy_tree:\n{}", tree);

    Ok(())
}

pub const fn chunk_key_encoding(&self) -> &ChunkKeyEncoding

Get the chunk key encoding.

pub const fn storage_transformers(&self) -> &StorageTransformerChain

Get the storage transformers.

pub const fn dimension_names(&self) -> &Option<Vec<DimensionName>>

Get the dimension names.

pub fn set_dimension_names( &mut self, dimension_names: Option<Vec<DimensionName>>, ) -> &mut Self

Set the dimension names.

pub const fn attributes(&self) -> &Map<String, Value>

Get the attributes.

pub fn attributes_mut(&mut self) -> &mut Map<String, Value>

Mutably borrow the array attributes.

pub fn metadata(&self) -> &ArrayMetadata

Return the underlying array metadata.

Examples found in repository

examples/custom_data_type_variable_size.rs (line 181)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let array = ArrayBuilder::new(
        vec![4, 1], // array shape
        vec![3, 1], // regular chunk shape
        Arc::new(CustomDataTypeVariableSize),
        [],
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data = [
        CustomDataTypeVariableSizeElement::from(Some(1.0)),
        CustomDataTypeVariableSizeElement::from(None),
        CustomDataTypeVariableSizeElement::from(Some(3.0)),
    ];
    array.store_chunk(&[0, 0], &data).unwrap();

    let data: Vec<CustomDataTypeVariableSizeElement> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    assert_eq!(data[0], CustomDataTypeVariableSizeElement::from(Some(1.0)));
    assert_eq!(data[1], CustomDataTypeVariableSizeElement::from(None));
    assert_eq!(data[2], CustomDataTypeVariableSizeElement::from(Some(3.0)));
    assert_eq!(data[3], CustomDataTypeVariableSizeElement::from(None));

    println!("{data:#?}");
}

examples/custom_data_type_fixed_size.rs (line 305)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let fill_value = CustomDataTypeFixedSizeElement { x: 1, y: 2.3 };
    let array = ArrayBuilder::new(
        vec![4, 1], // array shape
        vec![2, 1], // regular chunk shape
        Arc::new(CustomDataTypeFixedSize),
        FillValue::new(fill_value.to_ne_bytes().to_vec()),
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data = [
        CustomDataTypeFixedSizeElement { x: 3, y: 4.5 },
        CustomDataTypeFixedSizeElement { x: 6, y: 7.8 },
    ];
    array.store_chunk(&[0, 0], &data).unwrap();

    let data: Vec<CustomDataTypeFixedSizeElement> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    assert_eq!(data[0], CustomDataTypeFixedSizeElement { x: 3, y: 4.5 });
    assert_eq!(data[1], CustomDataTypeFixedSizeElement { x: 6, y: 7.8 });
    assert_eq!(data[2], CustomDataTypeFixedSizeElement { x: 1, y: 2.3 });
    assert_eq!(data[3], CustomDataTypeFixedSizeElement { x: 1, y: 2.3 });

    println!("{data:#?}");
}

examples/custom_data_type_uint12.rs (line 218)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let fill_value = CustomDataTypeUInt12Element::try_from(15).unwrap();
    let array = ArrayBuilder::new(
        vec![4096, 1], // array shape
        vec![5, 1],    // regular chunk shape
        Arc::new(CustomDataTypeUInt12),
        FillValue::new(fill_value.into_le_bytes().to_vec()),
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .array_to_bytes_codec(Arc::new(zarrs::array::codec::PackBitsCodec::default()))
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data: Vec<CustomDataTypeUInt12Element> = (0..4096)
        .map(|i| CustomDataTypeUInt12Element::try_from(i).unwrap())
        .collect();

    array
        .store_array_subset(&array.subset_all(), &data)
        .unwrap();

    let mut data: Vec<CustomDataTypeUInt12Element> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    for (i, d) in data.drain(0..4096).enumerate() {
        let element = CustomDataTypeUInt12Element::try_from(i as u64).unwrap();
        assert_eq!(d, element);
        let element_pd: Vec<CustomDataTypeUInt12Element> = array
            .retrieve_array_subset(&[(i as u64)..i as u64 + 1, 0..1])
            .unwrap();
        assert_eq!(element_pd[0], element);
    }
}

examples/custom_data_type_float8_e3m4.rs (line 228)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let fill_value = CustomDataTypeFloat8e3m4Element::from(1.23);
    let array = ArrayBuilder::new(
        vec![6, 1], // array shape
        vec![5, 1], // regular chunk shape
        Arc::new(CustomDataTypeFloat8e3m4),
        FillValue::new(fill_value.into_ne_bytes().to_vec()),
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data = [
        CustomDataTypeFloat8e3m4Element::from(2.34),
        CustomDataTypeFloat8e3m4Element::from(3.45),
        CustomDataTypeFloat8e3m4Element::from(f32::INFINITY),
        CustomDataTypeFloat8e3m4Element::from(f32::NEG_INFINITY),
        CustomDataTypeFloat8e3m4Element::from(f32::NAN),
    ];
    array.store_chunk(&[0, 0], &data).unwrap();

    let data: Vec<CustomDataTypeFloat8e3m4Element> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    for f in &data {
        println!(
            "float8_e3m4: {:08b} f32: {}",
            f.into_ne_bytes()[0],
            f.into_f32()
        );
    }

    assert_eq!(data[0], CustomDataTypeFloat8e3m4Element::from(2.34));
    assert_eq!(data[1], CustomDataTypeFloat8e3m4Element::from(3.45));
    assert_eq!(
        data[2],
        CustomDataTypeFloat8e3m4Element::from(f32::INFINITY)
    );
    assert_eq!(
        data[3],
        CustomDataTypeFloat8e3m4Element::from(f32::NEG_INFINITY)
    );
    assert_eq!(data[4], CustomDataTypeFloat8e3m4Element::from(f32::NAN));
    assert_eq!(data[5], CustomDataTypeFloat8e3m4Element::from(1.23));
}

examples/async_http_array_read.rs (line 54)

async fn http_array_read(backend: Backend) -> Result<(), Box<dyn std::error::Error>> {
    const HTTP_URL: &str =
        "https://raw.githubusercontent.com/zarrs/zarrs/main/zarrs/tests/data/array_write_read.zarr";
    const ARRAY_PATH: &str = "/group/array";

    // Create a HTTP store
    let mut store: AsyncReadableStorage = match backend {
        // Backend::OpenDAL => {
        //     let builder = opendal::services::Http::default().endpoint(HTTP_URL);
        //     let operator = opendal::Operator::new(builder)?.finish();
        //     Arc::new(zarrs_opendal::AsyncOpendalStore::new(operator))
        // }
        Backend::ObjectStore => {
            let options = object_store::ClientOptions::new().with_allow_http(true);
            let store = object_store::http::HttpBuilder::new()
                .with_url(HTTP_URL)
                .with_client_options(options)
                .build()?;
            Arc::new(zarrs_object_store::AsyncObjectStore::new(store))
        }
    };
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Init the existing array, reading metadata
    let array = Array::async_open(store, ARRAY_PATH).await?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Read the whole array
    let data_all: ArrayD<f32> = array
        .async_retrieve_array_subset(&array.subset_all())
        .await?;
    println!("The whole array is:\n{data_all}\n");

    // Read a chunk back from the store
    let chunk_indices = vec![1, 0];
    let data_chunk: ArrayD<f32> = array.async_retrieve_chunk(&chunk_indices).await?;
    println!("Chunk [1,0] is:\n{data_chunk}\n");

    // Read the central 4x2 subset of the array
    let subset_4x2 = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_4x2: ArrayD<f32> = array.async_retrieve_array_subset(&subset_4x2).await?;
    println!("The middle 4x2 subset is:\n{data_4x2}\n");

    Ok(())
}

examples/custom_data_type_uint4.rs (line 220)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let fill_value = CustomDataTypeUInt4Element::try_from(15).unwrap();
    let array = ArrayBuilder::new(
        vec![6, 1], // array shape
        vec![5, 1], // regular chunk shape
        Arc::new(CustomDataTypeUInt4),
        FillValue::new(fill_value.into_ne_bytes().to_vec()),
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .array_to_bytes_codec(Arc::new(zarrs::array::codec::PackBitsCodec::default()))
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data = [
        CustomDataTypeUInt4Element::try_from(1).unwrap(),
        CustomDataTypeUInt4Element::try_from(2).unwrap(),
        CustomDataTypeUInt4Element::try_from(3).unwrap(),
        CustomDataTypeUInt4Element::try_from(4).unwrap(),
        CustomDataTypeUInt4Element::try_from(5).unwrap(),
    ];
    array.store_chunk(&[0, 0], &data).unwrap();

    let data: Vec<CustomDataTypeUInt4Element> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    for f in &data {
        println!("uint4: {:08b} u8: {}", f.into_u8(), f.into_u8());
    }

    assert_eq!(data[0], CustomDataTypeUInt4Element::try_from(1).unwrap());
    assert_eq!(data[1], CustomDataTypeUInt4Element::try_from(2).unwrap());
    assert_eq!(data[2], CustomDataTypeUInt4Element::try_from(3).unwrap());
    assert_eq!(data[3], CustomDataTypeUInt4Element::try_from(4).unwrap());
    assert_eq!(data[4], CustomDataTypeUInt4Element::try_from(5).unwrap());
    assert_eq!(data[5], CustomDataTypeUInt4Element::try_from(15).unwrap());

    let data: Vec<CustomDataTypeUInt4Element> = array.retrieve_array_subset(&[1..3, 0..1]).unwrap();
    assert_eq!(data[0], CustomDataTypeUInt4Element::try_from(2).unwrap());
    assert_eq!(data[1], CustomDataTypeUInt4Element::try_from(3).unwrap());
}

Additional examples can be found in:

• examples/data_type_optional_nested.rs
• examples/sync_http_array_read.rs
• examples/array_write_read_string.rs
• examples/rectangular_array_write_read.rs
• examples/array_write_read.rs
• examples/sharded_array_write_read.rs
• examples/data_type_optional.rs
• examples/array_write_read_ndarray.rs
• examples/async_array_write_read.rs

pub fn metadata_opt(&self, options: &ArrayMetadataOptions) -> ArrayMetadata

Return a new ArrayMetadata with ArrayMetadataOptions applied.

This method is used internally by Array::store_metadata and Array::store_metadata_opt.

pub fn builder(&self) -> ArrayBuilder

Create an array builder matching the parameters of this array.

pub fn chunk_grid_shape(&self) -> &[u64]

Return the shape of the chunk grid (i.e., the number of chunks).

Examples found in repository

examples/data_type_optional.rs (line 87)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create an in-memory store
    // let store = Arc::new(zarrs::filesystem::FilesystemStore::new(
    //     "zarrs/tests/data/v3/array_optional.zarr",
    // )?);
    let store = Arc::new(zarrs::storage::store::MemoryStore::new());

    // Build the codec chains for the optional codec
    let array = ArrayBuilder::new(
        vec![4, 4],                       // 4x4 array
        vec![2, 2],                       // 2x2 chunks
        data_type::uint8().to_optional(), // Optional uint8
        FillValue::new_optional_null(),   // Null fill value: [0]
    )
    .dimension_names(["y", "x"].into())
    .attributes(
        serde_json::json!({
            "description": r#"A 4x4 array of optional uint8 values with some missing data.
N marks missing (`None`=`null`) values:
 0  N  2  3 
 N  5  N  7 
 8  9  N  N 
12  N  N  N"#,
        })
        .as_object()
        .unwrap()
        .clone(),
    )
    .build(store.clone(), "/array")?;
    array.store_metadata_opt(
        &zarrs::array::ArrayMetadataOptions::default().with_include_zarrs_metadata(false),
    )?;

    println!("Array metadata:\n{}", array.metadata().to_string_pretty());

    // Create some data with missing values
    let data = ndarray::array![
        [Some(0u8), None, Some(2u8), Some(3u8)],
        [None, Some(5u8), None, Some(7u8)],
        [Some(8u8), Some(9u8), None, None],
        [Some(12u8), None, None, None],
    ]
    .into_dyn();

    // Write the data
    array.store_array_subset(&array.subset_all(), data.clone())?;

    // Read back the data
    let data_read: ArrayD<Option<u8>> = array.retrieve_array_subset(&array.subset_all())?;

    // Verify data integrity
    assert_eq!(data, data_read);

    // Display the data in a grid format
    println!("Data grid, N marks missing (`None`=`null`) values");
    println!("   0  1  2  3");
    for y in 0..4 {
        print!("{} ", y);
        for x in 0..4 {
            match data_read[[y, x]] {
                Some(value) => print!("{:2} ", value),
                None => print!(" N "),
            }
        }
        println!();
    }

    // Print the raw bytes in all chunks
    println!("Raw bytes in all chunks:");
    let chunk_grid_shape = array.chunk_grid_shape();
    for chunk_y in 0..chunk_grid_shape[0] {
        for chunk_x in 0..chunk_grid_shape[1] {
            let chunk_indices = vec![chunk_y, chunk_x];
            let chunk_key = array.chunk_key(&chunk_indices);
            println!("  Chunk [{}, {}] (key: {}):", chunk_y, chunk_x, chunk_key);

            if let Some(chunk_bytes) = store.get(&chunk_key)? {
                println!("    Size: {} bytes", chunk_bytes.len());

                if chunk_bytes.len() >= 16 {
                    // Parse first 8 bytes as mask size (little-endian u64)
                    let mask_size = u64::from_le_bytes([
                        chunk_bytes[0],
                        chunk_bytes[1],
                        chunk_bytes[2],
                        chunk_bytes[3],
                        chunk_bytes[4],
                        chunk_bytes[5],
                        chunk_bytes[6],
                        chunk_bytes[7],
                    ]) as usize;

                    // Parse second 8 bytes as data size (little-endian u64)
                    let data_size = u64::from_le_bytes([
                        chunk_bytes[8],
                        chunk_bytes[9],
                        chunk_bytes[10],
                        chunk_bytes[11],
                        chunk_bytes[12],
                        chunk_bytes[13],
                        chunk_bytes[14],
                        chunk_bytes[15],
                    ]) as usize;

                    // Display mask size header with raw bytes
                    print!("    Mask size: 0b");
                    for byte in &chunk_bytes[0..8] {
                        print!("{:08b}", byte);
                    }
                    println!(" -> {} bytes", mask_size);

                    // Display data size header with raw bytes
                    print!("    Data size: 0b");
                    for byte in &chunk_bytes[8..16] {
                        print!("{:08b}", byte);
                    }
                    println!(" -> {} bytes", data_size);

                    // Show mask and data sections separately
                    if chunk_bytes.len() >= 16 + mask_size + data_size {
                        let mask_start = 16;
                        let data_start = 16 + mask_size;

                        // Show mask as binary
                        if mask_size > 0 {
                            println!("    Mask (binary):");
                            print!("      ");
                            for byte in &chunk_bytes[mask_start..mask_start + mask_size] {
                                print!("0b{:08b} ", byte);
                            }
                            println!();
                        }

                        // Show data as binary
                        if data_size > 0 {
                            println!("    Data (binary):");
                            print!("      ");
                            for byte in &chunk_bytes[data_start..data_start + data_size] {
                                print!("0b{:08b} ", byte);
                            }
                            println!();
                        }
                    }
                } else {
                    panic!("    Chunk too small to parse headers");
                }
            } else {
                println!("    Chunk missing (fill value chunk)");
            }
        }
    }
    Ok(())
}

pub fn chunk_key(&self, chunk_indices: &[u64]) -> StoreKey

Return the StoreKey of the chunk at chunk_indices.

Examples found in repository

examples/data_type_optional.rs (line 91)

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create an in-memory store
    // let store = Arc::new(zarrs::filesystem::FilesystemStore::new(
    //     "zarrs/tests/data/v3/array_optional.zarr",
    // )?);
    let store = Arc::new(zarrs::storage::store::MemoryStore::new());

    // Build the codec chains for the optional codec
    let array = ArrayBuilder::new(
        vec![4, 4],                       // 4x4 array
        vec![2, 2],                       // 2x2 chunks
        data_type::uint8().to_optional(), // Optional uint8
        FillValue::new_optional_null(),   // Null fill value: [0]
    )
    .dimension_names(["y", "x"].into())
    .attributes(
        serde_json::json!({
            "description": r#"A 4x4 array of optional uint8 values with some missing data.
N marks missing (`None`=`null`) values:
 0  N  2  3 
 N  5  N  7 
 8  9  N  N 
12  N  N  N"#,
        })
        .as_object()
        .unwrap()
        .clone(),
    )
    .build(store.clone(), "/array")?;
    array.store_metadata_opt(
        &zarrs::array::ArrayMetadataOptions::default().with_include_zarrs_metadata(false),
    )?;

    println!("Array metadata:\n{}", array.metadata().to_string_pretty());

    // Create some data with missing values
    let data = ndarray::array![
        [Some(0u8), None, Some(2u8), Some(3u8)],
        [None, Some(5u8), None, Some(7u8)],
        [Some(8u8), Some(9u8), None, None],
        [Some(12u8), None, None, None],
    ]
    .into_dyn();

    // Write the data
    array.store_array_subset(&array.subset_all(), data.clone())?;

    // Read back the data
    let data_read: ArrayD<Option<u8>> = array.retrieve_array_subset(&array.subset_all())?;

    // Verify data integrity
    assert_eq!(data, data_read);

    // Display the data in a grid format
    println!("Data grid, N marks missing (`None`=`null`) values");
    println!("   0  1  2  3");
    for y in 0..4 {
        print!("{} ", y);
        for x in 0..4 {
            match data_read[[y, x]] {
                Some(value) => print!("{:2} ", value),
                None => print!(" N "),
            }
        }
        println!();
    }

    // Print the raw bytes in all chunks
    println!("Raw bytes in all chunks:");
    let chunk_grid_shape = array.chunk_grid_shape();
    for chunk_y in 0..chunk_grid_shape[0] {
        for chunk_x in 0..chunk_grid_shape[1] {
            let chunk_indices = vec![chunk_y, chunk_x];
            let chunk_key = array.chunk_key(&chunk_indices);
            println!("  Chunk [{}, {}] (key: {}):", chunk_y, chunk_x, chunk_key);

            if let Some(chunk_bytes) = store.get(&chunk_key)? {
                println!("    Size: {} bytes", chunk_bytes.len());

                if chunk_bytes.len() >= 16 {
                    // Parse first 8 bytes as mask size (little-endian u64)
                    let mask_size = u64::from_le_bytes([
                        chunk_bytes[0],
                        chunk_bytes[1],
                        chunk_bytes[2],
                        chunk_bytes[3],
                        chunk_bytes[4],
                        chunk_bytes[5],
                        chunk_bytes[6],
                        chunk_bytes[7],
                    ]) as usize;

                    // Parse second 8 bytes as data size (little-endian u64)
                    let data_size = u64::from_le_bytes([
                        chunk_bytes[8],
                        chunk_bytes[9],
                        chunk_bytes[10],
                        chunk_bytes[11],
                        chunk_bytes[12],
                        chunk_bytes[13],
                        chunk_bytes[14],
                        chunk_bytes[15],
                    ]) as usize;

                    // Display mask size header with raw bytes
                    print!("    Mask size: 0b");
                    for byte in &chunk_bytes[0..8] {
                        print!("{:08b}", byte);
                    }
                    println!(" -> {} bytes", mask_size);

                    // Display data size header with raw bytes
                    print!("    Data size: 0b");
                    for byte in &chunk_bytes[8..16] {
                        print!("{:08b}", byte);
                    }
                    println!(" -> {} bytes", data_size);

                    // Show mask and data sections separately
                    if chunk_bytes.len() >= 16 + mask_size + data_size {
                        let mask_start = 16;
                        let data_start = 16 + mask_size;

                        // Show mask as binary
                        if mask_size > 0 {
                            println!("    Mask (binary):");
                            print!("      ");
                            for byte in &chunk_bytes[mask_start..mask_start + mask_size] {
                                print!("0b{:08b} ", byte);
                            }
                            println!();
                        }

                        // Show data as binary
                        if data_size > 0 {
                            println!("    Data (binary):");
                            print!("      ");
                            for byte in &chunk_bytes[data_start..data_start + data_size] {
                                print!("0b{:08b} ", byte);
                            }
                            println!();
                        }
                    }
                } else {
                    panic!("    Chunk too small to parse headers");
                }
            } else {
                println!("    Chunk missing (fill value chunk)");
            }
        }
    }
    Ok(())
}

pub fn chunk_origin( &self, chunk_indices: &[u64], ) -> Result<ArrayIndices, ArrayError>

Return the origin of the chunk at chunk_indices.

Errors

Returns ArrayError::InvalidChunkGridIndicesError if the chunk_indices are incompatible with the chunk grid.

pub fn chunk_shape( &self, chunk_indices: &[u64], ) -> Result<ChunkShape, ArrayError>

Return the shape of the chunk at chunk_indices.

Errors

Returns ArrayError::InvalidChunkGridIndicesError if the chunk_indices are incompatible with the chunk grid.

pub fn subset_all(&self) -> ArraySubset

Return an array subset that spans the entire array.

Examples found in repository

examples/custom_data_type_variable_size.rs (line 191)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let array = ArrayBuilder::new(
        vec![4, 1], // array shape
        vec![3, 1], // regular chunk shape
        Arc::new(CustomDataTypeVariableSize),
        [],
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data = [
        CustomDataTypeVariableSizeElement::from(Some(1.0)),
        CustomDataTypeVariableSizeElement::from(None),
        CustomDataTypeVariableSizeElement::from(Some(3.0)),
    ];
    array.store_chunk(&[0, 0], &data).unwrap();

    let data: Vec<CustomDataTypeVariableSizeElement> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    assert_eq!(data[0], CustomDataTypeVariableSizeElement::from(Some(1.0)));
    assert_eq!(data[1], CustomDataTypeVariableSizeElement::from(None));
    assert_eq!(data[2], CustomDataTypeVariableSizeElement::from(Some(3.0)));
    assert_eq!(data[3], CustomDataTypeVariableSizeElement::from(None));

    println!("{data:#?}");
}

examples/custom_data_type_fixed_size.rs (line 314)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let fill_value = CustomDataTypeFixedSizeElement { x: 1, y: 2.3 };
    let array = ArrayBuilder::new(
        vec![4, 1], // array shape
        vec![2, 1], // regular chunk shape
        Arc::new(CustomDataTypeFixedSize),
        FillValue::new(fill_value.to_ne_bytes().to_vec()),
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data = [
        CustomDataTypeFixedSizeElement { x: 3, y: 4.5 },
        CustomDataTypeFixedSizeElement { x: 6, y: 7.8 },
    ];
    array.store_chunk(&[0, 0], &data).unwrap();

    let data: Vec<CustomDataTypeFixedSizeElement> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    assert_eq!(data[0], CustomDataTypeFixedSizeElement { x: 3, y: 4.5 });
    assert_eq!(data[1], CustomDataTypeFixedSizeElement { x: 6, y: 7.8 });
    assert_eq!(data[2], CustomDataTypeFixedSizeElement { x: 1, y: 2.3 });
    assert_eq!(data[3], CustomDataTypeFixedSizeElement { x: 1, y: 2.3 });

    println!("{data:#?}");
}

examples/custom_data_type_uint12.rs (line 225)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let fill_value = CustomDataTypeUInt12Element::try_from(15).unwrap();
    let array = ArrayBuilder::new(
        vec![4096, 1], // array shape
        vec![5, 1],    // regular chunk shape
        Arc::new(CustomDataTypeUInt12),
        FillValue::new(fill_value.into_le_bytes().to_vec()),
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .array_to_bytes_codec(Arc::new(zarrs::array::codec::PackBitsCodec::default()))
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data: Vec<CustomDataTypeUInt12Element> = (0..4096)
        .map(|i| CustomDataTypeUInt12Element::try_from(i).unwrap())
        .collect();

    array
        .store_array_subset(&array.subset_all(), &data)
        .unwrap();

    let mut data: Vec<CustomDataTypeUInt12Element> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    for (i, d) in data.drain(0..4096).enumerate() {
        let element = CustomDataTypeUInt12Element::try_from(i as u64).unwrap();
        assert_eq!(d, element);
        let element_pd: Vec<CustomDataTypeUInt12Element> = array
            .retrieve_array_subset(&[(i as u64)..i as u64 + 1, 0..1])
            .unwrap();
        assert_eq!(element_pd[0], element);
    }
}

examples/custom_data_type_float8_e3m4.rs (line 240)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let fill_value = CustomDataTypeFloat8e3m4Element::from(1.23);
    let array = ArrayBuilder::new(
        vec![6, 1], // array shape
        vec![5, 1], // regular chunk shape
        Arc::new(CustomDataTypeFloat8e3m4),
        FillValue::new(fill_value.into_ne_bytes().to_vec()),
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data = [
        CustomDataTypeFloat8e3m4Element::from(2.34),
        CustomDataTypeFloat8e3m4Element::from(3.45),
        CustomDataTypeFloat8e3m4Element::from(f32::INFINITY),
        CustomDataTypeFloat8e3m4Element::from(f32::NEG_INFINITY),
        CustomDataTypeFloat8e3m4Element::from(f32::NAN),
    ];
    array.store_chunk(&[0, 0], &data).unwrap();

    let data: Vec<CustomDataTypeFloat8e3m4Element> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    for f in &data {
        println!(
            "float8_e3m4: {:08b} f32: {}",
            f.into_ne_bytes()[0],
            f.into_f32()
        );
    }

    assert_eq!(data[0], CustomDataTypeFloat8e3m4Element::from(2.34));
    assert_eq!(data[1], CustomDataTypeFloat8e3m4Element::from(3.45));
    assert_eq!(
        data[2],
        CustomDataTypeFloat8e3m4Element::from(f32::INFINITY)
    );
    assert_eq!(
        data[3],
        CustomDataTypeFloat8e3m4Element::from(f32::NEG_INFINITY)
    );
    assert_eq!(data[4], CustomDataTypeFloat8e3m4Element::from(f32::NAN));
    assert_eq!(data[5], CustomDataTypeFloat8e3m4Element::from(1.23));
}

examples/async_http_array_read.rs (line 59)

async fn http_array_read(backend: Backend) -> Result<(), Box<dyn std::error::Error>> {
    const HTTP_URL: &str =
        "https://raw.githubusercontent.com/zarrs/zarrs/main/zarrs/tests/data/array_write_read.zarr";
    const ARRAY_PATH: &str = "/group/array";

    // Create a HTTP store
    let mut store: AsyncReadableStorage = match backend {
        // Backend::OpenDAL => {
        //     let builder = opendal::services::Http::default().endpoint(HTTP_URL);
        //     let operator = opendal::Operator::new(builder)?.finish();
        //     Arc::new(zarrs_opendal::AsyncOpendalStore::new(operator))
        // }
        Backend::ObjectStore => {
            let options = object_store::ClientOptions::new().with_allow_http(true);
            let store = object_store::http::HttpBuilder::new()
                .with_url(HTTP_URL)
                .with_client_options(options)
                .build()?;
            Arc::new(zarrs_object_store::AsyncObjectStore::new(store))
        }
    };
    if let Some(arg1) = std::env::args().collect::<Vec<_>>().get(1)
        && arg1 == "--usage-log"
    {
        let log_writer = Arc::new(std::sync::Mutex::new(
            // std::io::BufWriter::new(
            std::io::stdout(),
            //    )
        ));
        store = Arc::new(UsageLogStorageAdapter::new(store, log_writer, || {
            chrono::Utc::now().format("[%T%.3f] ").to_string()
        }));
    }

    // Init the existing array, reading metadata
    let array = Array::async_open(store, ARRAY_PATH).await?;

    println!(
        "The array metadata is:\n{}\n",
        array.metadata().to_string_pretty()
    );

    // Read the whole array
    let data_all: ArrayD<f32> = array
        .async_retrieve_array_subset(&array.subset_all())
        .await?;
    println!("The whole array is:\n{data_all}\n");

    // Read a chunk back from the store
    let chunk_indices = vec![1, 0];
    let data_chunk: ArrayD<f32> = array.async_retrieve_chunk(&chunk_indices).await?;
    println!("Chunk [1,0] is:\n{data_chunk}\n");

    // Read the central 4x2 subset of the array
    let subset_4x2 = ArraySubset::new_with_ranges(&[2..6, 3..5]); // the center 4x2 region
    let data_4x2: ArrayD<f32> = array.async_retrieve_array_subset(&subset_4x2).await?;
    println!("The middle 4x2 subset is:\n{data_4x2}\n");

    Ok(())
}

examples/custom_data_type_uint4.rs (line 232)

fn main() {
    let store = std::sync::Arc::new(MemoryStore::default());
    let array_path = "/array";
    let fill_value = CustomDataTypeUInt4Element::try_from(15).unwrap();
    let array = ArrayBuilder::new(
        vec![6, 1], // array shape
        vec![5, 1], // regular chunk shape
        Arc::new(CustomDataTypeUInt4),
        FillValue::new(fill_value.into_ne_bytes().to_vec()),
    )
    .array_to_array_codecs(vec![
        #[cfg(feature = "transpose")]
        Arc::new(zarrs::array::codec::TransposeCodec::new(
            zarrs::array::codec::array_to_array::transpose::TransposeOrder::new(&[1, 0]).unwrap(),
        )),
    ])
    .array_to_bytes_codec(Arc::new(zarrs::array::codec::PackBitsCodec::default()))
    .bytes_to_bytes_codecs(vec![
        #[cfg(feature = "gzip")]
        Arc::new(zarrs::array::codec::GzipCodec::new(5).unwrap()),
        #[cfg(feature = "crc32c")]
        Arc::new(zarrs::array::codec::Crc32cCodec::new()),
    ])
    // .storage_transformers(vec![].into())
    .build(store, array_path)
    .unwrap();
    println!("{}", array.metadata().to_string_pretty());

    let data = [
        CustomDataTypeUInt4Element::try_from(1).unwrap(),
        CustomDataTypeUInt4Element::try_from(2).unwrap(),
        CustomDataTypeUInt4Element::try_from(3).unwrap(),
        CustomDataTypeUInt4Element::try_from(4).unwrap(),
        CustomDataTypeUInt4Element::try_from(5).unwrap(),
    ];
    array.store_chunk(&[0, 0], &data).unwrap();

    let data: Vec<CustomDataTypeUInt4Element> =
        array.retrieve_array_subset(&array.subset_all()).unwrap();

    for f in &data {
        println!("uint4: {:08b} u8: {}", f.into_u8(), f.into_u8());
    }

    assert_eq!(data[0], CustomDataTypeUInt4Element::try_from(1).unwrap());
    assert_eq!(data[1], CustomDataTypeUInt4Element::try_from(2).unwrap());
    assert_eq!(data[2], CustomDataTypeUInt4Element::try_from(3).unwrap());
    assert_eq!(data[3], CustomDataTypeUInt4Element::try_from(4).unwrap());
    assert_eq!(data[4], CustomDataTypeUInt4Element::try_from(5).unwrap());
    assert_eq!(data[5], CustomDataTypeUInt4Element::try_from(15).unwrap());

    let data: Vec<CustomDataTypeUInt4Element> = array.retrieve_array_subset(&[1..3, 0..1]).unwrap();
    assert_eq!(data[0], CustomDataTypeUInt4Element::try_from(2).unwrap());
    assert_eq!(data[1], CustomDataTypeUInt4Element::try_from(3).unwrap());
}

Additional examples can be found in:

• examples/data_type_optional_nested.rs
• examples/sync_http_array_read.rs
• examples/array_write_read_string.rs
• examples/rectangular_array_write_read.rs
• examples/array_write_read.rs
• examples/sharded_array_write_read.rs
• examples/data_type_optional.rs
• examples/array_write_read_ndarray.rs
• examples/async_array_write_read.rs

pub fn chunk_shape_usize( &self, chunk_indices: &[u64], ) -> Result<Vec<usize>, ArrayError>

Return the shape of the chunk at chunk_indices.

Errors

Returns ArrayError::InvalidChunkGridIndicesError if the chunk_indices are incompatible with the chunk grid.

Panics

Panics if any component of the chunk shape exceeds usize::MAX.

pub fn chunk_subset( &self, chunk_indices: &[u64], ) -> Result<ArraySubset, ArrayError>

Return the array subset of the chunk at chunk_indices.

Errors

Returns ArrayError::InvalidChunkGridIndicesError if the chunk_indices are incompatible with the chunk grid.

pub fn chunk_subset_bounded( &self, chunk_indices: &[u64], ) -> Result<ArraySubset, ArrayError>

Return the array subset of the chunk at chunk_indices bounded by the array shape.

Errors

Returns ArrayError::InvalidChunkGridIndicesError if the chunk_indices are incompatible with the chunk grid.

pub fn chunks_subset( &self, chunks: &dyn ArraySubsetTraits, ) -> Result<ArraySubset, ArrayError>

Return the array subset of chunks.

Errors

Returns ArrayError::InvalidChunkGridIndicesError if a chunk in chunks is incompatible with the chunk grid.

pub fn chunks_subset_bounded( &self, chunks: &dyn ArraySubsetTraits, ) -> Result<ArraySubset, ArrayError>

Return the array subset of chunks bounded by the array shape.

Errors

Returns ArrayError::InvalidChunkGridIndicesError if the chunk_indices are incompatible with the chunk grid.

pub fn chunks_in_array_subset( &self, array_subset: &dyn ArraySubsetTraits, ) -> Result<Option<ArraySubset>, IncompatibleDimensionalityError>

Return an array subset indicating the chunks intersecting array_subset.

Returns None if the intersecting chunks cannot be determined.

Errors

Returns IncompatibleDimensionalityError if the array subset has an incorrect dimensionality.

pub fn to_v3(self) -> Result<Self, ArrayMetadataV2ToV3Error>

Convert the array to Zarr V3.

Errors

Returns a ArrayMetadataV2ToV3Error if the metadata is not compatible with Zarr V3 metadata.

Trait Implementations

impl<TStorage: ?Sized> ArrayShardedExt for Array<TStorage>

Available on crate feature sharding only.

fn is_sharded(&self) -> bool

Returns true if the array to bytes codec of the array is sharding_indexed.

fn is_exclusively_sharded(&self) -> bool

Returns true if the array-to-bytes codec of the array is sharding_indexed and the array has no array-to-array or bytes-to-bytes codecs.

fn subchunk_shape(&self) -> Option<ChunkShape>

Return the subchunk shape as defined in the sharding_indexed codec metadata.

fn effective_subchunk_shape(&self) -> Option<ChunkShape>

The effective subchunk shape.

fn subchunk_grid(&self) -> ChunkGrid

Retrieve the subchunk grid.

fn subchunk_grid_shape(&self) -> ArrayShape

Return the shape of the subchunk grid (i.e., the number of subchunks).

impl<TStorage: ?Sized + ReadableStorageTraits + 'static> ArrayShardedReadableExt<TStorage> for Array<TStorage>

Available on crate feature sharding only.

fn subchunk_byte_range( &self, cache: &ArrayShardedReadableExtCache, subchunk_indices: &[u64], ) -> Result<Option<ByteRange>, ArrayError>

Retrieve the byte range of an encoded subchunk.

fn retrieve_encoded_subchunk( &self, cache: &ArrayShardedReadableExtCache, subchunk_indices: &[u64], ) -> Result<Option<Vec<u8>>, ArrayError>

Retrieve the encoded bytes of a subchunk.

fn retrieve_subchunk_opt<T: FromArrayBytes>( &self, cache: &ArrayShardedReadableExtCache, subchunk_indices: &[u64], options: &CodecOptions, ) -> Result<T, ArrayError>

Read and decode the subchunk at subchunk_indices into its bytes.

fn retrieve_subchunk_elements_opt<T: ElementOwned>( &self, cache: &ArrayShardedReadableExtCache, subchunk_indices: &[u64], options: &CodecOptions, ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_subchunk_opt::<Vec<T>>() instead

Read and decode the subchunk at subchunk_indices into a vector of its elements.

fn retrieve_subchunk_ndarray_opt<T: ElementOwned>( &self, cache: &ArrayShardedReadableExtCache, subchunk_indices: &[u64], options: &CodecOptions, ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_subchunk_opt::<ndarray::ArrayD<T>>() instead

Available on crate feature ndarray only.

Read and decode the subchunk at subchunk_indices into an ndarray::ArrayD.

fn retrieve_subchunks_opt<T: FromArrayBytes>( &self, cache: &ArrayShardedReadableExtCache, subchunks: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<T, ArrayError>

Read and decode the subchunks at subchunks.

fn retrieve_subchunks_elements_opt<T: ElementOwned>( &self, cache: &ArrayShardedReadableExtCache, subchunks: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_subchunks_opt::<Vec<T>>() instead

Read and decode the subchunks at subchunks into a vector of their elements.

fn retrieve_subchunks_ndarray_opt<T: ElementOwned>( &self, cache: &ArrayShardedReadableExtCache, subchunks: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_subchunks_opt::<ndarray::ArrayD<T>>() instead

Available on crate feature ndarray only.

Read and decode the subchunks at subchunks into an ndarray::ArrayD.

fn retrieve_array_subset_sharded_opt<T: FromArrayBytes>( &self, cache: &ArrayShardedReadableExtCache, array_subset: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<T, ArrayError>

Read and decode the array_subset of array.

fn retrieve_array_subset_elements_sharded_opt<T: ElementOwned>( &self, cache: &ArrayShardedReadableExtCache, array_subset: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<Vec<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_array_subset_sharded_opt::<Vec<T>>() instead

Read and decode the array_subset of array into a vector of its elements.

fn retrieve_array_subset_ndarray_sharded_opt<T: ElementOwned>( &self, cache: &ArrayShardedReadableExtCache, array_subset: &dyn ArraySubsetTraits, options: &CodecOptions, ) -> Result<ArrayD<T>, ArrayError>

👎Deprecated since 0.23.0: Use retrieve_array_subset_sharded_opt::<ndarray::ArrayD<T>>() instead

Available on crate feature ndarray only.

Read and decode the array_subset of array into an ndarray::ArrayD.

impl<TStorage: ?Sized + AsyncReadableStorageTraits + 'static> AsyncArrayShardedReadableExt<TStorage> for Array<TStorage>

Available on crate feature async only.

fn async_subchunk_byte_range<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, cache: &'life1 AsyncArrayShardedReadableExtCache, subchunk_indices: &'life2 [u64], ) -> Pin<Box<dyn Future<Output = Result<Option<ByteRange>, ArrayError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Retrieve the byte range of an encoded subchunk.

fn async_retrieve_encoded_subchunk<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, cache: &'life1 AsyncArrayShardedReadableExtCache, subchunk_indices: &'life2 [u64], ) -> Pin<Box<dyn Future<Output = Result<Option<Vec<u8>>, ArrayError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Retrieve the encoded bytes of a subchunk.

fn async_retrieve_subchunk_opt<'life0, 'life1, 'life2, 'life3, 'async_trait, T>( &'life0 self, cache: &'life1 AsyncArrayShardedReadableExtCache, subchunk_indices: &'life2 [u64], options: &'life3 CodecOptions, ) -> Pin<Box<dyn Future<Output = Result<T, ArrayError>> + Send + 'async_trait>>

where T: 'async_trait + FromArrayBytes + MaybeSend, Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Read and decode the subchunk at subchunk_indices into its bytes.

fn async_retrieve_subchunk_elements_opt<'life0, 'life1, 'life2, 'life3, 'async_trait, T>( &'life0 self, cache: &'life1 AsyncArrayShardedReadableExtCache, subchunk_indices: &'life2 [u64], options: &'life3 CodecOptions, ) -> Pin<Box<dyn Future<Output = Result<Vec<T>, ArrayError>> + Send + 'async_trait>>

where T: 'async_trait + ElementOwned + MaybeSend + MaybeSync, Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

👎Deprecated since 0.23.0: Use async_retrieve_subchunk_opt::<Vec<T>>() instead

Read and decode the subchunk at subchunk_indices into a vector of its elements.

fn async_retrieve_subchunk_ndarray_opt<'life0, 'life1, 'life2, 'life3, 'async_trait, T>( &'life0 self, cache: &'life1 AsyncArrayShardedReadableExtCache, subchunk_indices: &'life2 [u64], options: &'life3 CodecOptions, ) -> Pin<Box<dyn Future<Output = Result<ArrayD<T>, ArrayError>> + Send + 'async_trait>>

where T: 'async_trait + ElementOwned + MaybeSend + MaybeSync, Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

👎Deprecated since 0.23.0: Use async_retrieve_subchunk_opt::<ndarray::ArrayD<T>>() instead

Available on crate feature ndarray only.

Read and decode the subchunk at subchunk_indices into an ndarray::ArrayD.

fn async_retrieve_subchunks_opt<'life0, 'life1, 'life2, 'life3, 'async_trait, T>( &'life0 self, cache: &'life1 AsyncArrayShardedReadableExtCache, subchunks: &'life2 dyn ArraySubsetTraits, options: &'life3 CodecOptions, ) -> Pin<Box<dyn Future<Output = Result<T, ArrayError>> + Send + 'async_trait>>

where T: 'async_trait + FromArrayBytes + MaybeSend, Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Read and decode the chunks at chunks.

fn async_retrieve_subchunks_elements_opt<'life0, 'life1, 'life2, 'life3, 'async_trait, T>( &'life0 self, cache: &'life1 AsyncArrayShardedReadableExtCache, subchunks: &'life2 dyn ArraySubsetTraits, options: &'life3 CodecOptions, ) -> Pin<Box<dyn Future<Output = Result<Vec<T>, ArrayError>> + Send + 'async_trait>>

where T: 'async_trait + ElementOwned + MaybeSend + MaybeSync, Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

👎Deprecated since 0.23.0: Use async_retrieve_subchunks_opt::<Vec<T>>() instead

Read and decode the subchunks at subchunks into a vector of their elements.

fn async_retrieve_subchunks_ndarray_opt<'life0, 'life1, 'life2, 'life3, 'async_trait, T>( &'life0 self, cache: &'life1 AsyncArrayShardedReadableExtCache, subchunks: &'life2 dyn ArraySubsetTraits, options: &'life3 CodecOptions, ) -> Pin<Box<dyn Future<Output = Result<ArrayD<T>, ArrayError>> + Send + 'async_trait>>

where T: 'async_trait + ElementOwned + MaybeSend + MaybeSync, Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

👎Deprecated since 0.23.0: Use async_retrieve_subchunks_opt::<ndarray::ArrayD<T>>() instead

Available on crate feature ndarray only.

Read and decode the subchunks at subchunks into an ndarray::ArrayD.

fn async_retrieve_array_subset_sharded_opt<'life0, 'life1, 'life2, 'life3, 'async_trait, T>( &'life0 self, cache: &'life1 AsyncArrayShardedReadableExtCache, array_subset: &'life2 dyn ArraySubsetTraits, options: &'life3 CodecOptions, ) -> Pin<Box<dyn Future<Output = Result<T, ArrayError>> + Send + 'async_trait>>

where T: 'async_trait + FromArrayBytes + MaybeSend, Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Read and decode the array_subset of array.

fn async_retrieve_array_subset_elements_sharded_opt<'life0, 'life1, 'life2, 'life3, 'async_trait, T>( &'life0 self, cache: &'life1 AsyncArrayShardedReadableExtCache, array_subset: &'life2 dyn ArraySubsetTraits, options: &'life3 CodecOptions, ) -> Pin<Box<dyn Future<Output = Result<Vec<T>, ArrayError>> + Send + 'async_trait>>

where T: 'async_trait + ElementOwned + MaybeSend + MaybeSync, Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

👎Deprecated since 0.23.0: Use async_retrieve_array_subset_sharded_opt::<Vec<T>>() instead

Read and decode the array_subset of array into a vector of its elements.

fn async_retrieve_array_subset_ndarray_sharded_opt<'life0, 'life1, 'life2, 'life3, 'async_trait, T>( &'life0 self, cache: &'life1 AsyncArrayShardedReadableExtCache, array_subset: &'life2 dyn ArraySubsetTraits, options: &'life3 CodecOptions, ) -> Pin<Box<dyn Future<Output = Result<ArrayD<T>, ArrayError>> + Send + 'async_trait>>

where T: 'async_trait + ElementOwned + MaybeSend + MaybeSync, Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

👎Deprecated since 0.23.0: Use async_retrieve_array_subset_sharded_opt::<ndarray::ArrayD<T>>() instead

Available on crate feature ndarray only.

Read and decode the array_subset of array into an ndarray::ArrayD.

impl<TStorage: Debug + ?Sized> Debug for Array<TStorage>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<TStorage: ?Sized> TryFrom<&Array<TStorage>> for Hierarchy

type Error = NodeCreateError

The type returned in the event of a conversion error.

fn try_from(array: &Array<TStorage>) -> Result<Self, Self::Error>

Performs the conversion.

impl<TStorage: ?Sized> TryFrom<Array<TStorage>> for Hierarchy

type Error = NodeCreateError

The type returned in the event of a conversion error.

fn try_from(array: Array<TStorage>) -> Result<Self, Self::Error>

Performs the conversion.