Crate hdf5_pure 

Pure-Rust HDF5 file reading, writing, and in-place editing library.

hdf5-pure is a zero-C-dependency crate for creating, reading, and editing HDF5 files. It is WASM-compatible and supports no_std environments with alloc.

Writing files

use hdf5_pure::{FileBuilder, AttrValue};

let mut builder = FileBuilder::new();
builder.create_dataset("data")
    .with_f64_data(&[1.0, 2.0, 3.0])
    .with_shape(&[3])
    .set_attr("unit", AttrValue::String("m/s".into()));
let bytes = builder.finish().unwrap();

Reading files

use hdf5_pure::File;

let file = File::from_bytes(std::fs::read("output.h5").unwrap()).unwrap();
let ds = file.dataset("data").unwrap();
let values = ds.read_f64().unwrap();

Generic over the element type

The typed with_*_data / read_* methods have generic counterparts bounded by H5Element: DatasetBuilder::with_data writes a flat slice of any supported scalar and Dataset::read reads one back, so you can write code generic over the stored type.

use hdf5_pure::{File, FileBuilder, H5Element};

fn store<T: H5Element>(fb: &mut FileBuilder, name: &str, values: &[T]) {
    fb.create_dataset(name).with_data(values);
}

let mut fb = FileBuilder::new();
store(&mut fb, "counts", &[1u32, 2, 3]);
let file = File::from_bytes(fb.finish().unwrap()).unwrap();
let counts: Vec<u32> = file.dataset("counts").unwrap().read().unwrap();
assert_eq!(counts, vec![1, 2, 3]);

Editing files in place

EditSession opens an existing file and adds, deletes, copies, or overwrites objects without reading it all in and rewriting it. New data and rebuilt object headers are appended at end-of-file and the superblock is repointed last, so the cost is proportional to what changes rather than to the file size. It edits files written by this crate, the reference HDF5 C library, and h5py across all of their on-disk formats, and refuses — rather than silently degrade the file — anything it cannot reproduce faithfully.

use hdf5_pure::EditSession;

let mut session = EditSession::open("output.h5").unwrap();
session.create_dataset("extra").with_f64_data(&[4.0, 5.0]);
session.write_dataset("data").with_f64_data(&[7.0, 8.0]); // H5Dwrite (overwrite)
session.delete("old");             // H5Ldelete
session.commit().unwrap();

write_dataset overwrites an existing contiguous or compact dataset’s values; the replacement must match the on-disk datatype and shape (it is a value write, not a reshape/retype), and a same-length contiguous overwrite is applied straight into the existing data block without rewriting any header.

Streaming large files

File::open reads the whole file into memory. To read a file too large to buffer (for example a multi-gigabyte file on a 32-bit host, where it exceeds the address space), use File::open_streaming, which fetches metadata and dataset chunks from the file on demand instead of buffering it whole. The reading API is identical; only the backing store differs. Reads of contiguous, compact, and chunked datasets are supported; the streaming backend currently resolves only latest-format (v2) groups and does not yet read attributes.

use hdf5_pure::File;

let file = File::open_streaming("huge.h5").unwrap();
let values = file.dataset("signal").unwrap().read_f64().unwrap();

N-dimensional arrays (ndarray feature)

With the ndarray feature, datasets can be written from and read back into [ndarray] arrays of any rank, in row-major (C) order:

use hdf5_pure::{File, FileBuilder};
use ndarray::{array, Array2};

let a: Array2<f64> = array![[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]];
let mut fb = FileBuilder::new();
fb.create_dataset("m").with_ndarray(&a);
let bytes = fb.finish().unwrap();

let file = File::from_bytes(bytes).unwrap();
let back: Array2<f64> = file.dataset("m").unwrap().read_array().unwrap();
assert_eq!(a, back);

Modules

mat

MATLAB v7.3 (.mat) file conventions on top of HDF5.

Structs

ChunkCacheConfig

Configuration for a per-dataset chunk cache.

ChunkCacheStats

A read-only snapshot of a dataset’s chunk-cache occupancy.

CompoundMember

A member of a compound datatype.

CompoundTypeBuilder

Builder for constructing HDF5 compound (struct) datatypes.

Dataset

A lightweight handle to an HDF5 dataset.

DatasetAccessOptions

Dataset-access options applied when opening a single dataset.

DatasetBuilder

Builder for datasets.

EditSession

An open HDF5 file being edited in place.

EnumMember

A member of an enumeration datatype.

EnumTypeBuilder

Builder for constructing HDF5 enumeration datatypes.

ExplicitCompoundTypeBuilder

Builder for an HDF5 compound datatype with explicit field offsets and size.

File

An open HDF5 file for reading.

FileAccessOptions

File-access options applied when opening an HDF5 file.

FileBuilder

Builder for creating a new HDF5 file.

FileSpaceInfo

A parsed (or to-be-written) File Space Info message.

FinishedGroup

A finished group ready for the file writer.

Group

A lightweight handle to an HDF5 group.

GroupBuilder

Builder for HDF5 groups.

MetadataCacheConfig

Initial metadata-cache settings for streaming file access.

RepackOptions

Options controlling a repack.

SwmrWriter

An append writer over an existing HDF5 file.

VlenStringReadOptions

Allocation limits for reading variable-length strings.

Enums

AttrValue

Convenient attribute values for the write API.

CharacterSet

Character set encoding.

DType

Simplified datatype enum for the high-level API.

Datatype

Parsed HDF5 datatype.

DatatypeByteOrder

Byte order of numeric data.

Error

Errors that can occur when using the high-level API.

FileLocking

Policy for OS advisory file locking when opening a file for editing.

FileSpaceStrategy

File-space management strategy, mirroring HDF5’s H5F_fspace_strategy_t (set with H5Pset_file_space_strategy).

FormatError

Errors that can occur when parsing HDF5 binary format structures.

LibVer

A library-version boundary, mirroring HDF5’s H5F_libver_t.

Object

The resolved target of an HDF5 object reference (H5R_OBJECT): either a group or a dataset.

ReferenceType

Reference type.

ScaleOffset

Scale-offset compression mode requested by the writer.

StringPadding

String padding type.

Traits

CompoundField

A type that can occupy one field of an HDF5 compound value.

CompoundType

A safely encoded HDF5 compound element.

H5Element

A Rust scalar type that can be stored as an HDF5 dataset element.

Functions

is_hdf5

Test whether a file looks like an HDF5 file, without reading it whole.

is_hdf5_bytes

Test whether an in-memory buffer begins (at a permitted offset) with the HDF5 signature. The buffer-backed counterpart of is_hdf5.

make_f32_type

make_f64_type

make_i8_type

make_i16_type

make_i32_type

make_i64_type

make_u8_type

make_u16_type

make_u32_type

make_u64_type

repack

Repack src into a new file at dst, applying options.