zarrs_tools

Various tools for creating and manipulating Zarr v3 data with the zarrs rust crate.

Changelog (CHANGELOG.md)

Tools

• zarrs_reencode: reencode a Zarr v3 array. Example.
  • Can change the chunk size, shard size, compression, etc.
  • Suitable for round trip benchmarking
• zarrs_binary2zarr: create a Zarr v3 array from piped binary data. Example.
• zarrs_ncvar2zarr (requires ncvar2zarr feature): convert a netCDF variable to a zarr V3 array. Example.
• zarrs_benchmark_read_sync: Measure the time to read (decode) each chunk of an array using the zarrs sync API.
• zarrs_benchmark_read_async: Measure the time to read (decode) each chunk of an array using the zarrs async API.

See docs/benchmarks.md for some benchmark measurements.

Install

From crates.io

cargo install zarrs_tools

From source

cargo install --path .

Enabling SIMD intrinsics

Encoding and decoding performance may be improved with avx2/sse2 enabled (if supported).

This can be enabled by compiling with either of:

• RUSTFLAGS="-C target-cpu=native"
• RUSTFLAGS="-C target-feature=+avx2,+sse2"

Enabling non-default zarrs codecs

Non-default zarrs codecs (see zarrs crate features) can be enabled by passing them as feature flags.

For example:

cargo install zarrs_tools --features zarrs/bitround,zarrs/zfp

Licence

zarrs_tools is licensed under either of

• the Apache License, Version 2.0 LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0 or
• the MIT license LICENSE-MIT or http://opensource.org/licenses/MIT, at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.