Crate medh5 

medh5

The MEDH5 format engine. A .medh5 file is one subject at one or more timepoints, with every image, annotation, transform and curation record about them, in one self-describing HDF5 container. This crate is the canonical implementation of formats 1.0 and 1.1 (the specification, and 1.1 for the optional clinical profile): the data model, HDF5 I/O, validation, chunked access, compression, geometry and transforms, annotations, provenance, integrity — and, for 1.1, clinical events on a subject clock, strict prospective selection, and the task and cache contract training builds on.

It has two siblings, and all three read and write the same bytes:

• the Python package (pip install medh5), a thin layer over this crate that adds the NumPy, PyTorch, MONAI and converter integrations;
• the medh5 command line (medh5-cli), a native binary over this crate.

HDF5 and C-Blosc2 are built from source and linked statically (see medh5-sys), so a dependent crate needs no HDF5 installation. For an MSVC target, set CFLAGS_x86_64_pc_windows_msvc=-DNDEBUG (or the variable for your target): without it HDF5 keeps its assertions, and medh5-sys stops the build.

Write and read a sample

use medh5::array::NdArray;
use medh5::sample::{create, open_sample, GridOptions, ImageOptions};

fn main() -> medh5::Result<()> {
    let path = std::env::temp_dir().join("medh5-readme-case_0001.medh5");

    // A writer validates every call and commits atomically: a reader never
    // sees half a file, and a crash leaves the previous one intact.
    let ct = NdArray::from_vec(&[8, 16, 16], vec![0i16; 8 * 16 * 16])?;
    let mut w = create(&path, Some("case_0001"), Some("DEMO-0001"), "balanced", &[])?;
    w.add_grid("ct", &[8, 16, 16], &[2.0, 0.8, 0.8], GridOptions::default())?;
    w.add_image("CT", &ct, "ct", "CT", ImageOptions::default())?;
    let content_id = w.commit(true)?; // the Merkle root over every stored digest

    let sample = open_sample(&path)?;
    assert_eq!(sample.identity()?.subject_id, "DEMO-0001");
    assert_eq!(sample.content_id()?, content_id);
    let voxels = sample.image("CT")?.read(None, false, None)?;
    assert_eq!(voxels.shape(), [8, 16, 16]);
    assert!(sample.verify(None)?.ok());

    // The validator the writer is held to (§15); `strict` would also ask for a
    // de-identification record (W903).
    let report = medh5::validate::validate_file(&path, "integrity", None)?;
    assert!(report.ok(), "{:?}", report.codes());

    std::fs::remove_file(&path)?;
    Ok(())
}

Where things are

Module	Specification
sample	§2, §4, §14.4 — reading (Sample) and writing (SampleWriter, create, amend)
document	§2.4 — the sample document, everything in /meta
ids	§2.2, §2.3 — identifier rules
geometry	§3 — grids, affines, frames, multiscale
labels	§5 — label sets, hierarchies and ontology codes
annotations	§6–§9 — the voxel and geometric encodings, headers, coverage
transforms	§10 — affine, displacement, B-spline and composite transforms
curation	§11, §12 — identity, provenance, quality, timelines, splits, scrubbing
integrity, digest	§13 — per-object digests and content_id; the hash algorithms and digest strings
storage	§14 — codec profiles, chunking, the sampling index
validate, codes	§15 — the validator and its diagnostic code table
collection	§2.2 — packing samples into a .medh5c shard
convert	Appendix B — the 0.x migration, and the ConversionReport every importer fills
array	NdArray, the array every reader returns and every writer takes
dataset, sampling	cohort manifests, splits, statistics; patch sampling
bench	reproducing the performance targets
rng	the generator behind sampling, the same stream in every frontend
error	the one Error type, and the code each error carries
conformance	the conformance corpus third-party implementations run
version	1.1 §2 — which versions are read fully, read as a projection, or refused
clinical	1.1 §3–§10 — the clinical profile: records, columns, checks, selection, augmentation
companion	task-cache-1 — task manifests, source pins, preflight row views, feature caches
h5, json, numeric, pyval	plumbing, outside semver (below)

h5, json, numeric and pyval are the engine’s plumbing rather than the format, and from 2.1 they are outside semver: a minor release may change them. They are HDF5 access below the high-level API, the format’s exact JSON serialisations, NumPy-compatible reductions and the 1.x value coercions. The documented way to the canonical JSON of §5.1 is labels::canonical_json; an attribute value is AttrValue, re-exported at the crate root.

The crate’s public signatures take and return types from three other crates, so they are public dependencies: hdf5-metno 0.15 (re-exported as hdf5), ndarray 0.17 (re-exported as ndarray) and IndexMap 2. A bump of any of them is a major medh5 release.

Errors are one Error type whose diagnostic code, when it has one, is the code the validator reports for the same defect (§15.2, 1.1 §11.2).

Re-exports

pub use error::Error;

pub use error::Result;

pub use h5::attrs::AttrValue;

pub use hdf5;

pub use ndarray;

Modules

annotations

Annotations: one coherent unit of ground truth per group (spec §6–§9).

array

Dense arrays of any element type the format stores.

bench

Reproducing the performance targets on the reader’s own hardware.

clinical

The clinical profile (format 1.1): clinical context beside the images.

codes

The normative diagnostic code table (spec 1.0 §15.2, extended by 1.1 §11.2).

collection

Collections — many sample roots in one file (spec §2.2).

companion

The task-and-cache companion contract (docs/spec/task-cache-1.md).

conformance

The conformance corpus and the suite that publishes it (spec §15).

convert

Conversion infrastructure shared by every importer, and the 0.x migration.

curation

Curation records: who produced what, how good it is, and who it is about.

dataset

Cohort tools: manifests, splits, streaming statistics, cross-file checks.

digest

Hash algorithms and digest strings (spec §2.1, §13).

document

The sample document: everything in /meta (spec §2.4).

error

Errors raised by the engine.

geometry

Geometry: grids, the index-world affine, and multiscale pyramids (spec §3, §4.3).

h5

HDF5 plumbing: attribute codecs, dynamic-dtype datasets, atomic create, copy-on-write and the low-level operations the high-level API lacks.

ids

Identifier rules (spec §2.2, §2.3).

integrity

Content addressing and verification (spec §13).

json

JSON exactly as the format’s documents have always been serialized.

labels

Label sets: id -> meaning (spec §5).

numeric

Reductions that round the way NumPy’s do.

pyval

Coercions of JSON values, as the 1.x reference implementation applied them.

rng

Randomness behind patch sampling, index subsampling and synthetic data.

sample

Samples: reading (Sample), images, and writing (SampleWriter).

sampling

Patch and timepoint-pair sampling (§14.3).

storage

Storage-layer concerns: chunking, codecs and derived sampling indices (spec §14).

transforms

Registration transforms (spec §10).

validate

Validation (spec §15).

version

Format versions: what this engine reads, validates and writes (1.1 §2).

Macros

bail_coded

Return early with a coded validation error.

bail_invalid

Return early with an uncoded validation error.

with_array

Run $body with $a bound to the typed array inside an NdArray.

with_dtype

Run $body with $T bound to the Rust element type of a DType.

Constants

FORMAT_VERSION

The newest MEDH5 format version this engine implements.

VERSION

The package version, stamped into every file’s generator attribute.