Crate hdf5_pure 

Pure-Rust HDF5 file reading, writing, and in-place editing library.

hdf5-pure is a zero-C-dependency crate for creating, reading, and editing HDF5 files. It is WASM-compatible and supports no_std environments with alloc.

Writing files

use hdf5_pure::{FileBuilder, AttrValue};

let mut builder = FileBuilder::new();
builder.create_dataset("data")
    .with_f64_data(&[1.0, 2.0, 3.0])
    .with_shape(&[3])
    .set_attr("unit", AttrValue::String("m/s".into()));
let bytes = builder.finish().unwrap();

Reading files

use hdf5_pure::File;

let file = File::from_bytes(std::fs::read("output.h5").unwrap()).unwrap();
let ds = file.dataset("data").unwrap();
let values = ds.read_f64().unwrap();

Generic over the element type

The typed with_*_data / read_* methods have generic counterparts bounded by H5Element: DatasetBuilder::with_data writes a flat slice of any supported scalar and Dataset::read reads one back, so you can write code generic over the stored type.

use hdf5_pure::{File, FileBuilder, H5Element};

fn store<T: H5Element>(fb: &mut FileBuilder, name: &str, values: &[T]) {
    fb.create_dataset(name).with_data(values);
}

let mut fb = FileBuilder::new();
store(&mut fb, "counts", &[1u32, 2, 3]);
let file = File::from_bytes(fb.finish().unwrap()).unwrap();
let counts: Vec<u32> = file.dataset("counts").unwrap().read().unwrap();
assert_eq!(counts, vec![1, 2, 3]);

Editing files in place

Open an existing file for reading and writing with File::open_rw and reach every object by name through owned Dataset and Group handles that add, delete, copy, or overwrite objects without rewriting the file from scratch. New data and rebuilt object headers are appended at end-of-file and the superblock is repointed last, so the cost is proportional to what changes rather than to the file size. It edits files written by this crate, the reference HDF5 C library, and h5py across all of their on-disk formats, and refuses — rather than silently degrade the file — anything it cannot reproduce faithfully. Memory follows the file rather than the caller’s choice of function: a latest-format file with no userblock is edited without ever building a whole-file copy of it (see MemoryStrategy).

use hdf5_pure::File;

let file = File::open_rw("output.h5").unwrap();
let root = file.root();
root.create_dataset("extra", |b| { b.with_f64_data(&[4.0, 5.0]); }).unwrap();
file.dataset("data").unwrap().write(&[7.0, 8.0]).unwrap(); // H5Dwrite (overwrite)
root.delete("old").unwrap();                               // H5Ldelete
file.commit().unwrap();                                    // apply staged edits

A value overwrite (Dataset::write) must match the on-disk datatype and shape (it is a value write, not a reshape/retype); a same-length contiguous overwrite is applied straight into the existing data block without rewriting any header. Immediate, amortized-O(1) row appends use Dataset::append and need no commit.

Streaming large files

File::open reads the whole file into memory. To read a file too large to buffer (for example a multi-gigabyte file on a 32-bit host, where it exceeds the address space), use File::open_streaming, which fetches metadata and dataset chunks from the file on demand instead of buffering it whole. The reading API is identical; only the backing store differs. Reads of contiguous, compact, and chunked datasets are supported; the streaming backend currently resolves only latest-format (v2) groups and does not yet read attributes.

use hdf5_pure::File;

let file = File::open_streaming("huge.h5").unwrap();
let values = file.dataset("signal").unwrap().read_f64().unwrap();

N-dimensional arrays (ndarray feature)

With the ndarray feature, datasets can be written from and read back into ndarray arrays of any rank, in row-major (C) order:

use hdf5_pure::{File, FileBuilder};
use ndarray::{array, Array2};

let a: Array2<f64> = array![[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]];
let mut fb = FileBuilder::new();
fb.create_dataset("m").with_ndarray(&a);
let bytes = fb.finish().unwrap();

let file = File::from_bytes(bytes).unwrap();
let back: Array2<f64> = file.dataset("m").unwrap().read_array().unwrap();
assert_eq!(a, back);

Modules

mat

MATLAB v7.3 (.mat) file conventions on top of HDF5.

Structs

AppendBuilder

Accumulates elements to append to an existing chunked, unlimited dataset via Dataset::append_staged, in call order along the dataset’s first (axis-0) dimension.

Chunk

The location and on-disk footprint of one stored chunk.

ChunkCacheConfig

Configuration for a per-dataset chunk cache.

ChunkCacheStats

A read-only snapshot of a dataset’s chunk-cache occupancy.

CompoundMember

A member of a compound datatype.

CompoundTypeBuilder

Builder for constructing HDF5 compound (struct) datatypes.

Dataset

An owned handle to an HDF5 dataset.

DatasetAccessProperties

Dataset-access properties applied when opening a single dataset.

DatasetBuilder

Builder for datasets.

EnumMember

A member of an enumeration datatype.

EnumTypeBuilder

Builder for constructing HDF5 enumeration datatypes.

ExplicitCompoundTypeBuilder

Builder for an HDF5 compound datatype with explicit field offsets and size.

File

An open HDF5 file.

FileAccessProperties

File-access properties applied when opening an HDF5 file.

FileBuilder

Builder for creating a new HDF5 file.

FileCreateProperties

File-creation properties applied when writing a new HDF5 file.

FileSpaceInfo

A parsed (or to-be-written) File Space Info message.

Filter

One filter in a dataset’s pipeline.

FinishedGroup

A finished group ready for the file writer.

Group

An owned handle to an HDF5 group.

GroupBuilder

Builder for HDF5 groups.

MetadataCacheConfig

Initial metadata-cache settings for streaming file access.

RepackOptions

Options controlling a repack.

SpaceAccounting

A snapshot of a writable file’s live space usage (issue #150).

StagedGroup

A group that exists only as a staged edit, handed to Group::create_group_with’s closure so attributes can be set on a group that is not yet committed (and so has no resolvable header to hang a Group handle off).

VlenStringReadOptions

Allocation limits for reading variable-length strings.

Enums

AttrValue

Convenient attribute values for the write API.

CharacterSet

Character set encoding.

ChunkIndex

The kind of index a chunked dataset uses to locate its chunks.

DType

Simplified datatype enum for the high-level API.

Datatype

Parsed HDF5 datatype.

DatatypeByteOrder

Byte order of numeric data.

EditBacking

Which of the two read-write backends a file’s editing session is actually using, from File::edit_backing.

Error

Errors that can occur when using the high-level API.

FileLocking

Policy for OS advisory file locking when opening a file for editing.

FileSpaceStrategy

File-space management strategy, mirroring HDF5’s H5F_fspace_strategy_t (set with H5Pset_file_space_strategy).

FormatError

Errors that can occur when parsing HDF5 binary format structures.

Layout

How a dataset’s raw data is arranged on disk.

LibVer

A library-version boundary, mirroring HDF5’s H5F_libver_t.

MemoryStrategy

How much memory a read-write open may use to hold the file being edited.

Object

The resolved target of an HDF5 object reference (H5R_OBJECT): either a group or a dataset.

ReferenceType

Reference type.

ScaleOffset

Scale-offset compression mode requested by the writer.

StringPadding

String padding type.

VerifyResult

Outcome of checking a dataset against its stored provenance hash.

Constants

OBJECT_HEADER_MESSAGE_MAX

The largest message a version 2 object header can describe: its per-message size field is 2 bytes wide. A message past this must be refused rather than written with a truncated length (see FormatError::ObjectHeaderMessageTooLarge).

Traits

CompoundField

A type that can occupy one field of an HDF5 compound value.

CompoundType

A safely encoded HDF5 compound element.

H5Element

A Rust scalar type that can be stored as an HDF5 dataset element.

Functions

is_hdf5

Test whether a file looks like an HDF5 file, without reading it whole.

is_hdf5_bytes

Test whether an in-memory buffer begins (at a permitted offset) with the HDF5 signature. The buffer-backed counterpart of is_hdf5.

make_f32_type

make_f64_type

make_i8_type

make_i16_type

make_i32_type

make_i64_type

make_u8_type

make_u16_type

make_u32_type

make_u64_type

repack

Repack src into a new file at dst, applying options.

Type Aliases

DatasetAccessOptionsDeprecated

Former name of DatasetAccessProperties.

FileAccessOptionsDeprecated

Former name of FileAccessProperties.

FileCreateOptionsDeprecated

Former name of FileCreateProperties.