Crate zarrs_icechunk 

icechunk store support for the zarrs crate.

Icechunk is a transactional store that enables git-like version control of Zarr hierarchies.

zarrs_icechunk can read data in a range of archival formats (e.g., netCDF4, HDF5, etc.) that are converted to icechunk-backed “virtual Zarr datacubes” via VirtualiZarr (example below).

Version Compatibility Matrix

zarrs_icechunk	icechunk	zarrs (zarrs_storage)
0.4.x	0.3.x	0.22+ (0.4.x)
0.3.x	0.3.x	0.18+ (0.3.x)
0.2.x	0.2.x	0.18+ (0.3.x)
0.1.1	0.1.2+	0.18+ (0.3.x)
0.1.0	0.1.0-0.1.1	0.18+ (0.3.x)
0.1.0-alpha.3	0.1.0-alpha.13-	0.18+ (0.3.x)
0.1.0-alpha.2	0.1.0-alpha.8-12	0.18+ (0.3.x)
0.1.0-alpha.1	0.1.0-alpha.5-7	0.18+ (0.3.x)
0.1.0-alpha.0	0.1.0-alpha.3	0.17 (0.2.2)

Examples

Basic Usage and Version Control

use icechunk::{Repository, RepositoryConfig, repository::VersionInfo};
use zarrs_icechunk::AsyncIcechunkStore;
// Create an icechunk repository
let storage = icechunk::new_in_memory_storage().await?;
let config = RepositoryConfig::default();
let repo = Repository::create(Some(config), storage, HashMap::new()).await?;

// Do some array/metadata manipulation with zarrs, then commit a snapshot
let session = repo.writable_session("main").await?;
let store = Arc::new(AsyncIcechunkStore::new(session));
let snapshot0 = store.session().write().await.commit("Initial commit", None).await?;

// Do some more array/metadata manipulation, then commit another snapshot
let session = repo.writable_session("main").await?;
let store = Arc::new(AsyncIcechunkStore::new(session));
let snapshot1 = store.session().write().await.commit("Update data", None).await?;

// Checkout the first snapshot
let session = repo.readonly_session(&VersionInfo::SnapshotId(snapshot0)).await?;
let store = Arc::new(AsyncIcechunkStore::new(session));

Virtualise NetCDF as Zarr (via VirtualiZarr)

Decode a virtual Zarr array /examples/data/test.icechunk.zarr:

cargo run --example virtualizarr_netcdf

This references /examples/data/test[0,1].nc hosted in this repository over HTTP. /examples/data/test.icechunk.zarr was created with /examples/virtualizarr_netcdf.py.

Licence

zarrs_icechunk is licensed under either of

• the Apache License, Version 2.0 LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0 or
• the MIT license LICENSE-MIT or http://opensource.org/licenses/MIT, at your option.

Re-exports

pub use icechunk;

Structs

AsyncIcechunkStore

An asynchronous store backed by an icechunk::session::Session.