medh5 2.0.3

The MEDH5 format engine: a self-describing HDF5 container for one medical imaging subject and all of its ground truth
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118

# medh5

The MEDH5 format engine.  A `.medh5` file is **one subject at one or more
timepoints**, with every image, annotation, transform and curation record about
them, in one self-describing HDF5 container.  This crate is the canonical
implementation of formats 1.0 and 1.1 ([the specification], and [1.1] for the
optional clinical profile): the data model, HDF5 I/O, validation, chunked
access, compression, geometry and transforms, annotations, provenance,
integrity — and, for 1.1, clinical events on a subject clock, strict
prospective selection, and the [task and cache contract] training builds on.

It has two siblings, and all three read and write the same bytes:

- the **Python package** (`pip install medh5`), a thin layer over this crate
  that adds the NumPy, PyTorch, MONAI and converter integrations;
- the **`medh5` command line** ([`medh5-cli`]), a native binary over this crate.

HDF5 and C-Blosc2 are built from source and linked statically (see
[`medh5-sys`]), so a dependent crate needs no HDF5 installation.  For an MSVC
target, set `CFLAGS_x86_64_pc_windows_msvc=-DNDEBUG` (or the variable for your
target): without it HDF5 keeps its assertions, and `medh5-sys` stops the build.

## Write and read a sample

```rust
use medh5::array::NdArray;
use medh5::sample::{create, open_sample, GridOptions, ImageOptions};

fn main() -> medh5::Result<()> {
    let path = std::env::temp_dir().join("medh5-readme-case_0001.medh5");

    // A writer validates every call and commits atomically: a reader never
    // sees half a file, and a crash leaves the previous one intact.
    let ct = NdArray::from_vec(&[8, 16, 16], vec![0i16; 8 * 16 * 16])?;
    let mut w = create(&path, Some("case_0001"), Some("DEMO-0001"), "balanced", &[])?;
    w.add_grid("ct", &[8, 16, 16], &[2.0, 0.8, 0.8], GridOptions::default())?;
    w.add_image("CT", &ct, "ct", "CT", ImageOptions::default())?;
    let content_id = w.commit(true)?; // the Merkle root over every stored digest

    let sample = open_sample(&path)?;
    assert_eq!(sample.identity()?.subject_id, "DEMO-0001");
    assert_eq!(sample.content_id()?, content_id);
    let voxels = sample.image("CT")?.read(None, false, None)?;
    assert_eq!(voxels.shape(), [8, 16, 16]);
    assert!(sample.verify(None)?.ok());

    // The validator the writer is held to (§15); `strict` would also ask for a
    // de-identification record (W903).
    let report = medh5::validate::validate_file(&path, "integrity", None)?;
    assert!(report.ok(), "{:?}", report.codes());

    std::fs::remove_file(&path)?;
    Ok(())
}
```

## Where things are

| Module | Specification |
|---|---|
| [`sample`] | §2, §4, §14.4 — reading (`Sample`) and writing (`SampleWriter`, `create`, `amend`) |
| [`document`] | §2.4 — the sample document, everything in `/meta` |
| [`ids`] | §2.2, §2.3 — identifier rules |
| [`geometry`] | §3 — grids, affines, frames, multiscale |
| [`labels`] | §5 — label sets, hierarchies and ontology codes |
| [`annotations`] | §6–§9 — the voxel and geometric encodings, headers, coverage |
| [`transforms`] | §10 — affine, displacement, B-spline and composite transforms |
| [`curation`] | §11, §12 — identity, provenance, quality, timelines, splits, scrubbing |
| [`integrity`], [`digest`] | §13 — per-object digests and `content_id`; the hash algorithms and digest strings |
| [`storage`] | §14 — codec profiles, chunking, the sampling index |
| [`validate`] | §15 — the validator and its diagnostic codes ([`codes`]) |
| [`collection`] | §2.2 — packing samples into a `.medh5c` shard |
| [`convert`] | Appendix B — the 0.x migration, and the `ConversionReport` every importer fills |
| [`array`] | `NdArray`, the array every reader returns and every writer takes |
| [`dataset`], [`sampling`] | cohort manifests, splits, statistics; patch sampling |
| [`bench`] | reproducing the performance targets |
| [`conformance`] | the conformance corpus third-party implementations run |
| [`version`] | 1.1 §2 — which versions are read fully, read as a projection, or refused |
| [`clinical`] | 1.1 §3–§10 — the clinical profile: records, columns, checks, selection, augmentation |
| [`companion`] | task-cache-1 — task manifests, source pins, preflight row views, feature caches |

`h5`, `json`, `numeric`, `pyval` and `rng` are public too, and are the engine's
plumbing rather than the format: HDF5 access below the high-level API, the
format's exact JSON serialisations, NumPy-compatible reductions, the 1.x value
coercions, and the generator behind sampling.

Errors are one [`Error`] type whose diagnostic code, when it has one, is the
code the validator reports for the same defect (§15.2, [1.1] §11.2).

[the specification]: https://medh5.readthedocs.io/en/stable/spec/medh5-1.0/
[1.1]: https://medh5.readthedocs.io/en/stable/spec/medh5-1.1/
[task and cache contract]: https://medh5.readthedocs.io/en/stable/spec/task-cache-1/
[`version`]: https://docs.rs/medh5/latest/medh5/version/
[`clinical`]: https://docs.rs/medh5/latest/medh5/clinical/
[`companion`]: https://docs.rs/medh5/latest/medh5/companion/
[`medh5-cli`]: https://crates.io/crates/medh5-cli
[`medh5-sys`]: https://crates.io/crates/medh5-sys
[`sample`]: https://docs.rs/medh5/latest/medh5/sample/
[`document`]: https://docs.rs/medh5/latest/medh5/document/
[`ids`]: https://docs.rs/medh5/latest/medh5/ids/
[`digest`]: https://docs.rs/medh5/latest/medh5/digest/
[`convert`]: https://docs.rs/medh5/latest/medh5/convert/
[`array`]: https://docs.rs/medh5/latest/medh5/array/
[`bench`]: https://docs.rs/medh5/latest/medh5/bench/
[`geometry`]: https://docs.rs/medh5/latest/medh5/geometry/
[`annotations`]: https://docs.rs/medh5/latest/medh5/annotations/
[`transforms`]: https://docs.rs/medh5/latest/medh5/transforms/
[`labels`]: https://docs.rs/medh5/latest/medh5/labels/
[`curation`]: https://docs.rs/medh5/latest/medh5/curation/
[`integrity`]: https://docs.rs/medh5/latest/medh5/integrity/
[`storage`]: https://docs.rs/medh5/latest/medh5/storage/
[`validate`]: https://docs.rs/medh5/latest/medh5/validate/
[`codes`]: https://docs.rs/medh5/latest/medh5/codes/
[`collection`]: https://docs.rs/medh5/latest/medh5/collection/
[`dataset`]: https://docs.rs/medh5/latest/medh5/dataset/
[`sampling`]: https://docs.rs/medh5/latest/medh5/sampling/
[`conformance`]: https://docs.rs/medh5/latest/medh5/conformance/
[`Error`]: https://docs.rs/medh5/latest/medh5/enum.Error.html