hdf5-rust 2.0.0

Pure-Rust HDF5 reader and writer. No libhdf5, no C FFI, no crate dependencies.
Documentation

hdf5-rust

Pure-Rust HDF5 reader and writer. No libhdf5. No C FFI. No crate dependencies. Current release: 2.0.0 on crates.io.

Files are standard HDF5: h5py, MATLAB h5read, HDFView, and Julia HDF5.jl can open what this crate writes, and this crate can open what those tools write (contiguous IEEE, integer, and opaque; uncompressed chunked IEEE).

Inventory: capabilities. Patches: CONTRIBUTING.md.

Why this crate exists

Every other Rust HDF5 crate binds the C library. That blocks WebAssembly, no_std, reproducible builds, and any project that refuses foreign I/O kernels. Scientific code still needs the file format. hdf5-rust implements the on-disk specification in Rust.

What it does

  • Write superblock version 2, compact groups, contiguous datasets.
  • Read superblock versions 0, 1, 2, and 3; object headers v1 and v2; old-style symbol-table groups and new-style compact links.
  • Datatypes: IEEE_F64LE / IEEE_F32LE, signed/unsigned integers (8/16/32/64-bit), opaque records. Big-endian numerics are normalized to LE bit lanes on read. Compound types report HDF5DType::Compound with compound_fields + read_raw. Other classes remain HDF5DType::Other.
  • Dataspace: simple rank 0 (scalar) through 32 (the HDF5 format maximum). Images, volumes, and time series (rank 3–5) are normal, not an error. A rank-3 cube does not prevent reading a rank-2 table in the same file.
  • Chunked datasets: B-tree v1, B-tree v2, implicit, single-chunk, fixed-array, and unpaged extensible-array indexes (layout v3–v5). Filters applied in pure Rust: gzip/deflate, shuffle, Fletcher32, atomic n-bit, and integer scale-offset. Still unsupported, and returned as named errors: SZIP, floating-point scale-offset, n-bit array/compound types, and paged extensible-array data blocks. Never a panic and never guessed values.

IEEE values are stored as integer bit patterns (u64 / u32 lanes). This crate never uses hardware f32/f64 arithmetic. Callers wrap their own array types at the boundary.

Written datasets carry string attributes hdf5-rust-version and created.

Install

[dependencies]
hdf5-rust = "2.0.0"

std (file I/O) is on by default. For no_std + alloc:

hdf5-rust = { version = "2.0.0", default-features = false }

Then use Hdf5File::from_bytes / to_bytes.

MSRV: Rust 1.99 (rust-version in Cargo.toml; the repo pins it in rust-toolchain.toml).

Quick start

use hdf5_rust::Hdf5File;

fn main() -> hdf5_rust::Result<()> {
    // Integer IEEE binary64 bit patterns (1.0, 2.0, 3.0, 4.0, 5.0, 6.0).
    let bits: &[u64] = &[
        0x3FF0_0000_0000_0000,
        0x4000_0000_0000_0000,
        0x4008_0000_0000_0000,
        0x4010_0000_0000_0000,
        0x4014_0000_0000_0000,
        0x4018_0000_0000_0000,
    ];

    let mut file = Hdf5File::create();
    file.create_group("results")?;
    file.write_f64("results/data", &[2, 3], bits)?;
    file.write_i32("results/counts", &[2, 3], &[1, 2, 3, 4, 5, 6])?;

    let (shape, got) = file.read_f64("results/data")?;
    assert_eq!(shape, vec![2, 3]);
    assert_eq!(got, bits);

    let bytes = file.to_bytes()?;
    assert_eq!(&bytes[..8], &hdf5_rust::HDF5_SIGNATURE);
    Ok(())
}

save / open require the std feature (on by default). write_ieee64 / read_ieee64 are the same methods under names that do not contain hardware float tokens.

Credits

Atomic n-bit decompression, little-endian integer scale-offset unpack, and Fletcher32 (including the checksum accepted for files written before HDF5 1.6.3) are derived from the HDF5 library (H5Znbit.c, H5Zscaleoffset.c, H5checksum.c, H5Zfletcher32.c). Copyright 2006 The HDF Group. Copyright 1998–2006 The Board of Trustees of the University of Illinois. 3-clause BSD. The Huffman decode loop Huff::decode is derived from puff.c 2.3, Copyright (C) 2002–2013 Mark Adler, zlib license, and is marked as altered. The copyright and permission text is in NOTICE.

Acknowledgments

This crate was developed with help from AI assistants by SpaceXAI, Anthropic, OpenAI, Google, and Meta.

License

MIT OR Apache-2.0. See CONTRIBUTING.md if you want to report a bug or send a patch. The NOTICE file is part of the package and must be kept with the derived portions above.

HDF5 trademark

HDF5 is a trademark of The HDF Group. This crate is an independent implementation of the published file format. It is not affiliated with The HDF Group.