zarrs::array::codec::array_to_bytes

Module sharding

Source
Available on crate feature sharding only.
Expand description

The sharding array to bytes codec (Core).

Sharding logically splits chunks (shards) into sub-chunks (inner chunks) that can be individually compressed and accessed. This allows to colocate multiple chunks within one storage object, bundling them in shards.

This codec requires the sharding feature, which is enabled by default.

The ShardingCodecBuilder can help with creating a ShardingCodec.

§Compatible Implementations

This is a core codec and should be compatible with all Zarr V3 implementations that support it.

§Specification

§Codec name Aliases (Zarr V3)

  • sharding_indexed

§Codec configuration Example - ShardingCodecConfiguration:

{
    "chunk_shape": [32, 32, 32],
    "codecs": [
        {
            "name": "endian",
            "configuration": {
                "endian": "little"
            }
        },
        {
            "name": "gzip",
            "configuration": {
                "level": 1
            }
        }
    ],
    "index_codecs": [
        {
            "name": "endian",
            "configuration": {
                "endian": "little"
            }
        },
        { "name": "crc32c" }
    ]
}

Structs§

ShardingCodec
A sharding codec implementation.
ShardingCodecBuilder
A ShardingCodec builder.
ShardingCodecConfigurationV1
Sharding codec configuration parameters.

Enums§

ShardingCodecConfiguration
A wrapper to handle various versions of Sharding codec configuration parameters.
ShardingIndexLocation
The sharding index location.