zarrs_icechunk 0.5.1

icechunk store support for the zarrs crate
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

# zarrs_icechunk

[![Latest Version](https://img.shields.io/crates/v/zarrs_icechunk.svg)](https://crates.io/crates/zarrs_icechunk)
[![icechunk 0.3](https://img.shields.io/badge/icechunk-0.3-blue)](https://crates.io/crates/icechunk)
[![zarrs_icechunk documentation](https://docs.rs/zarrs_icechunk/badge.svg)](https://docs.rs/zarrs_icechunk)
![msrv](https://img.shields.io/crates/msrv/zarrs_icechunk)
[![build](https://github.com/LDeakin/zarrs_icechunk/actions/workflows/ci.yml/badge.svg)](https://github.com/LDeakin/zarrs_icechunk/actions/workflows/ci.yml)

[`icechunk`](https://crates.io/crates/icechunk) store support for the [`zarrs`](https://crates.io/crates/zarrs) Rust crate.

Icechunk is a transactional store that enables `git`-like version control of Zarr hierarchies.

`zarrs_icechunk` can read data in a range of archival formats (e.g., [`netCDF4`](https://www.unidata.ucar.edu/software/netcdf/), [`HDF5`](https://www.hdfgroup.org/solutions/hdf5/), etc.) that are converted to `icechunk`-backed "virtual Zarr datacubes" via [`VirtualiZarr`](https://github.com/zarr-developers/VirtualiZarr) (example below).

## Version Compatibility Matrix
See [doc/version_compatibility_matrix.md](./doc/version_compatibility_matrix.md).

## MSRV Policy
The minimum supported Rust version (MSRV) is the MSRV of [`icechunk`](https://crates.io/crates/icechunk).
The MSRV may be incremented in minor releases.

## Examples
### Basic Usage and Version Control
```rust
use icechunk::{Repository, RepositoryConfig, repository::VersionInfo};
use zarrs_icechunk::AsyncIcechunkStore;

// Create an icechunk repository
let storage = icechunk::new_in_memory_storage().await?;
let config = RepositoryConfig::default();
let repo = Repository::create(Some(config), storage, HashMap::new()).await?;

// Do some array/metadata manipulation with zarrs, then commit a snapshot
let session = repo.writable_session("main").await?;
let store = Arc::new(AsyncIcechunkStore::new(session));
let array: Array<AsyncIcechunkStore> = ...;
let snapshot0 = store.session().write().await.commit("Initial commit", None).await?;

// Do some more array/metadata manipulation, then commit another snapshot
let session = repo.writable_session("main").await?;
let store = Arc::new(AsyncIcechunkStore::new(session));
let array: Array<AsyncIcechunkStore> = ...;
let snapshot1 = store.session().write().await.commit("Update data", None).await?;

// Checkout the first snapshot
let session = repo.readonly_session(&VersionInfo::SnapshotId(snapshot0)).await?;
let store = Arc::new(AsyncIcechunkStore::new(session));
let array: Array<AsyncIcechunkStore> = ...;
```

### Virtualise NetCDF as Zarr (via [`VirtualiZarr`](https://github.com/zarr-developers/VirtualiZarr))
Decode a virtual Zarr array [`/examples/data/test.icechunk.zarr`]:
```bash
cargo run --example virtualizarr_netcdf
```
This references `/examples/data/test[0,1].nc` hosted in this repository over HTTP.
[`/examples/data/test.icechunk.zarr`] was created with [`/examples/virtualizarr_netcdf.py`](./examples/virtualizarr_netcdf.py).

## Licence
`zarrs_icechunk` is licensed under either of
 - the Apache License, Version 2.0 [LICENSE-APACHE](./LICENCE-APACHE) or <http://www.apache.org/licenses/LICENSE-2.0> or
 - the MIT license [LICENSE-MIT](./LICENCE-MIT) or <http://opensource.org/licenses/MIT>, at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.


[`/examples/data/test.icechunk.zarr`]: ./examples/data/test.icechunk.zarr