Crate zarrs_n5 

zarrs_n5

N5 support for the zarrs ecosystem.

Limitations

• Read-only
• No partial chunk reading
• “default” chunk mode (i.e. not varlen or object)
• Compression support:
  • N5 core
    • gzip
    • bzip2
    • lz4
    • xz
  • N5 extensions
    • zstd https://github.com/JaneliaSciComp/n5-zstandard
    • blosc https://github.com/saalfeldlab/n5-blosc
    • jpeg is implemented in java but not well documented
      • PRs welcome but I’m unlikely to prioritise this unless a Zarr JPEG codec were stabilised
• The handling of edge chunks is quite inefficient
• Zarr groups must have metadata documents, but N5 groups may not. This may lead to unexpected behaviour when discovering hierarchy structure. This library allows inferring a group with empty attributes when a metadata document is missing.
• N5 hierarchies have a root (metadata contains the "n5" key), but Zarr hierarchies do not; in practice this doesn’t matter much

Prior art

• constantinpape/z5: C++/Python reader over a common subset of Zarr and N5
• aschampion/rust-n5: rust implementation of N5
• zarr-developers/zarr-python: Zarr v2’s N5 support
• zarr-developers/n5py: python plugins for reading N5 through Zarr v3
• saalfeldlab/n5-zarr: Java implementation reading zarr data through the N5 interface

Usage

This crate is comprises

• N5Store, which wraps other zarrs stores
  • implements reading and listing, blocking and async, as supported by the wrapped store
• N5Codec, an array-to-bytes codec which handles the N5 block header, bigendian byte order, block data transposition, and compression
  • varlen and object chunk modes are not supported
  • not all N5 compressors are supported
• convert_n5_node and convert_n5_hierarchy, which adds Zarr metadata to N5 objects to allow them to be read as Zarr without the N5Store wrapper
  • this functionality is experimental and relies on unstable Zarr extensions which may not be supported by other implementations

When zarr.json (the usual home of Zarr metadata) is requested from the N5Store, it is read from the corresponding N5 attributes.json and converted to Zarr v3 metadata on the fly. This converted metadata contains configuration for the N5-specific chunk key encoding and codec plugins, so regular zarrs APIs can be used transparently.

Re-exports

pub use zarrs;

Structs

N5ArrayMetadata

Representation of N5 array metadata.

N5BlockHeader

Representation of the N5 block header.

N5Codec

Codec for N5 blocks.

N5CodecConfiguration

Configuration for N5Codec, which serializes compression configuration, if any.

N5GroupMetadata

Representation of N5 group metadata.

N5Store

An N5 store wrapping another Zarr store, which handles converting metadata.

Enums

Error

zarrs_n5 error type.

N5BlockMode

Mode of the N5 block.

N5Compression

N5 block compression configuration.

N5Metadata

Representation of N5 metadata, either an array or a group.

Functions

convert_n5_hierarchy

Convert an N5 hierarchy to a Zarr v3 hierarchy by reading the N5 metadata for each node and writing the corresponding Zarr metadata.

convert_n5_node

Convert an N5 node to a Zarr v3 node by reading the N5 metadata and writing the corresponding Zarr metadata.

Type Aliases

Result