Struct SwmrFileWriter 

pub struct SwmrFileWriter { /* private fields */ }

SWMR writer for streaming frame-based data to an HDF5 file.

Usage:

use rust_hdf5::swmr::SwmrFileWriter;

let mut writer = SwmrFileWriter::create("stream.h5").unwrap();
let ds = writer.create_streaming_dataset::<f32>("frames", &[256, 256]).unwrap();
writer.start_swmr().unwrap();

// Write frames
let frame_data = vec![0.0f32; 256 * 256];
let raw: Vec<u8> = frame_data.iter()
    .flat_map(|v| v.to_le_bytes())
    .collect();
writer.append_frame(ds, &raw).unwrap();
writer.flush().unwrap();

writer.close().unwrap();

Implementations

impl SwmrFileWriter

pub fn create<P: AsRef<Path>>(path: P) -> Result<Self>

Create a new HDF5 file for SWMR streaming using the env-var-derived locking policy.

pub fn create_with_locking<P: AsRef<Path>>( path: P, locking: FileLocking, ) -> Result<Self>

Create a new HDF5 file for SWMR streaming with an explicit locking policy. The writer holds an exclusive lock until Self::start_swmr is called, at which point the lock is downgraded to shared so concurrent SWMR readers can attach.

pub fn open_append<P: AsRef<Path>>(path: P) -> Result<Self>

Reopen a cleanly-closed HDF5 file to resume SWMR streaming.

Existing datasets are reconstructed; locate them with dataset_index, call start_swmr to re-enter SWMR mode, then continue with append_frame. Appending to a multi-frame-chunk dataset (chunk[0] > 1) after reopen is rejected — its final partial band was zero-padded at the original close. Recovering a crashed (never cleanly closed) file is not supported.

pub fn open_append_with_locking<P: AsRef<Path>>( path: P, locking: FileLocking, ) -> Result<Self>

Reopen a cleanly-closed HDF5 file to resume SWMR streaming with an explicit locking policy. See Self::open_append.

pub fn dataset_index(&self, name: &str) -> Option<usize>

Return the index of a dataset by name, or None if absent.

Mainly used after open_append to recover the index of a reconstructed dataset for append_frame.

pub fn create_streaming_dataset<T: H5Type>( &mut self, name: &str, frame_dims: &[u64], ) -> Result<usize>

Create a streaming dataset.

The dataset will have shape [0, frame_dims...] initially, with chunk dimensions [1, frame_dims...] and unlimited first dimension.

Returns the dataset index for use with append_frame.

pub fn create_streaming_dataset_with_datatype( &mut self, name: &str, datatype: DatatypeMessage, frame_dims: &[u64], ) -> Result<usize>

Self::create_streaming_dataset with an explicit stored datatype instead of a carrier T, for a type no T describes (e.g. a reduced-precision type for FilterPipeline::nbit); an element is datatype.element_size() bytes.

pub fn create_streaming_dataset_compressed<T: H5Type>( &mut self, name: &str, frame_dims: &[u64], pipeline: FilterPipeline, ) -> Result<usize>

Create a streaming dataset whose frames are compressed.

Like create_streaming_dataset but each appended frame is run through pipeline (e.g. FilterPipeline::deflate(4)). SWMR appends and in-place header updates work the same as for uncompressed streaming datasets.

pub fn create_streaming_dataset_compressed_with_datatype( &mut self, name: &str, datatype: DatatypeMessage, frame_dims: &[u64], pipeline: FilterPipeline, ) -> Result<usize>

Self::create_streaming_dataset_compressed with an explicit stored datatype instead of a carrier T, for a type no T describes (e.g. a reduced-precision type for FilterPipeline::nbit); an element is datatype.element_size() bytes.

pub fn create_streaming_dataset_tiled<T: H5Type>( &mut self, name: &str, frame_dims: &[u64], frame_chunk: &[u64], ) -> Result<usize>

Create a streaming dataset whose frames are split into fixed-size chunk tiles.

frame_dims is the per-frame shape (e.g. [1024, 1024]); frame_chunk is the tile shape within a frame (e.g. [256, 256]), of the same rank. The on-disk chunk shape becomes [1, frame_chunk...], so each frame is stored as product(frame_dims / frame_chunk) chunks instead of one. This mirrors area-detector tiling controls such as NDFileHDF5’s nRowChunks / nColChunks: it changes only the partial-read granularity and compression unit, not the stored data. append_frame accepts a whole frame and splits it into tiles automatically.

pub fn create_streaming_dataset_tiled_compressed<T: H5Type>( &mut self, name: &str, frame_dims: &[u64], frame_chunk: &[u64], pipeline: FilterPipeline, ) -> Result<usize>

Create a compressed streaming dataset whose frames are split into fixed-size chunk tiles. See create_streaming_dataset_tiled for the meaning of frame_chunk; each tile is the compression unit.

pub fn create_streaming_dataset_tiled_compressed_with_datatype( &mut self, name: &str, datatype: DatatypeMessage, frame_dims: &[u64], frame_chunk: &[u64], pipeline: FilterPipeline, ) -> Result<usize>

Self::create_streaming_dataset_tiled_compressed with an explicit stored datatype instead of a carrier T, for a type no T describes (e.g. a reduced-precision type for FilterPipeline::nbit); an element is datatype.element_size() bytes.

pub fn create_streaming_dataset_chunked<T: H5Type>( &mut self, name: &str, frame_dims: &[u64], chunk: &[u64], ) -> Result<usize>

Create a streaming dataset with full control over the chunk shape, including the frame axis.

chunk is the complete per-chunk shape, of rank frame_dims.len() + 1: chunk[0] frames per chunk (the NDFileHDF5 nFramesChunks control) and chunk[1..] the per-frame tile shape (nRowChunks / nColChunks). When chunk[0] > 1, append_frame buffers whole frames until a chunk band fills; the final partial band is written (zero-padded) at close, and the dataset’s logical frame count always equals the exact number of frames appended.

pub fn create_streaming_dataset_chunked_compressed<T: H5Type>( &mut self, name: &str, frame_dims: &[u64], chunk: &[u64], pipeline: FilterPipeline, ) -> Result<usize>

Compressed variant of create_streaming_dataset_chunked; each chunk is filtered independently through pipeline.

pub fn create_streaming_dataset_chunked_compressed_with_datatype( &mut self, name: &str, datatype: DatatypeMessage, frame_dims: &[u64], chunk: &[u64], pipeline: FilterPipeline, ) -> Result<usize>

Self::create_streaming_dataset_chunked_compressed with an explicit stored datatype instead of a carrier T, for a type no T describes (e.g. a reduced-precision type for FilterPipeline::nbit); an element is datatype.element_size() bytes.

pub fn create_streaming_dataset_with_max_frames( &mut self, name: &str, datatype: DatatypeMessage, frame_dims: &[u64], chunk: &[u64], max_frames: Option<u64>, pipeline: Option<FilterPipeline>, ) -> Result<usize>

Self::create_streaming_dataset_chunked with a frame axis that holds at most max_frames frames (None: unlimited) and an explicit stored datatype, filtered through pipeline when one is given: NDFileHDF5’s maxdims[0] = NumCapture. A bounded frame axis is indexed by a fixed array, the index libhdf5 picks for it, and an append past max_frames is refused.

pub fn create_grid_dataset<T: H5Type>( &mut self, name: &str, dims: &[u64], chunk: &[u64], ) -> Result<usize>

Create a fixed-shape multi-dimensional grid dataset that fills at explicit positions as frames arrive.

Unlike create_streaming_dataset, which appends frames along a single unlimited leading axis, this creates a dataset of the full bounded shape dims (no unlimited axis) and lets you place each frame at an arbitrary chunk position with write_chunk_at. This mirrors AreaDetector’s “extra dimensions” layout, where a scan of known size (e.g. [Na, Nb, H, W]) is filled in odometer order. chunk is the per-chunk shape of the same rank (typically [1, …, 1, H, W]). dims is also the maximum shape; set_extent moves the extent within it. Returns the dataset index.

pub fn create_grid_dataset_with_datatype( &mut self, name: &str, datatype: DatatypeMessage, dims: &[u64], chunk: &[u64], ) -> Result<usize>

Self::create_grid_dataset with an explicit stored datatype instead of a carrier T, for a type no T describes (e.g. a reduced-precision type for FilterPipeline::nbit); an element is datatype.element_size() bytes.

pub fn create_grid_dataset_compressed<T: H5Type>( &mut self, name: &str, dims: &[u64], chunk: &[u64], pipeline: FilterPipeline, ) -> Result<usize>

Compressed variant of create_grid_dataset; each chunk is filtered independently through pipeline, and write_chunk_raw_at can place chunks a codec already compressed.

pub fn create_grid_dataset_compressed_with_datatype( &mut self, name: &str, datatype: DatatypeMessage, dims: &[u64], chunk: &[u64], pipeline: FilterPipeline, ) -> Result<usize>

Self::create_grid_dataset_compressed with an explicit stored datatype instead of a carrier T, for a type no T describes (e.g. a reduced-precision type for FilterPipeline::nbit); an element is datatype.element_size() bytes.

pub fn create_hard_link( &mut self, parent_group_path: &str, link_name: &str, target_path: &str, ) -> Result<()>

Create a hard link: an additional name for a dataset or group that already exists in the file.

No data is copied — the link and its target share one object header, exactly as h5py / libhdf5 hard links do. This is the NeXus-style way to expose a streaming dataset at an aliased path.

• parent_group_path — full path of the group that will hold the link ("/" for the root group).
• link_name — leaf name of the new link within that group.
• target_path — full path of an existing dataset or group.

Visibility relative to SWMR mode

A link created before start_swmr is committed by start_swmr and is visible to SWMR readers for the whole streaming window. A link created after start_swmr is committed only by close; it does not appear to readers that attach during the live SWMR window. Create layout links before start_swmr when readers must resolve them while streaming.

pub fn create_group( &mut self, parent_group_path: &str, name: &str, ) -> Result<()>

Create a group in the file hierarchy.

• parent_group_path — full path of the parent group ("/" for the root group).
• name — leaf name of the new group.

A nested NeXus layout is built one level at a time, parent first:

writer.create_group("/", "entry").unwrap();
writer.create_group("/entry", "data").unwrap();

Like create_hard_link, a group created before start_swmr is visible to SWMR readers for the whole streaming window; one created after is committed only by close.

pub fn set_group_attr_string( &mut self, group_path: &str, name: &str, value: &str, ) -> Result<()>

Set a string attribute on a group, or on the root group when group_path is "/".

This is the NeXus way to tag a group with its class — for example set_group_attr_string("/entry", "NX_class", "NXentry"). An existing attribute of the same name is replaced.

After start_swmr a set or replace is committed at close, as libhdf5 1.14 commits it under SWMR; until then readers see the attributes as published. This holds for every attribute setter.

pub fn set_group_attr_numeric<T: H5Type>( &mut self, group_path: &str, name: &str, value: &T, ) -> Result<()>

Set a numeric scalar attribute on a group, or on the root group when group_path is "/". An existing attribute of the same name is replaced. After start_swmr it is committed at close — see set_group_attr_string.

pub fn set_group_attr_array<T: H5Type>( &mut self, group_path: &str, name: &str, dims: &[u64], values: &[T], ) -> Result<()>

Set a numeric array attribute on a group, or on the root group when group_path is "/" — the group form of set_dataset_attr_array, with the same shape rule. After start_swmr it is committed at close — see set_group_attr_string.

pub fn set_group_attr_typed( &mut self, group_path: &str, name: &str, datatype: DatatypeMessage, value: Vec<u8>, ) -> Result<()>

Set a scalar attribute of a caller-supplied datatype on a group, or on the root group when group_path is "/" — the SWMR form of H5Group::set_attr_typed. After start_swmr it is committed at close — see set_group_attr_string.

pub fn write_dataset<T: H5Type>( &mut self, name: &str, dims: &[u64], data: &[T], ) -> Result<usize>

Create a fixed-shape (non-streaming) dataset and write all its data in one call. Returns the dataset index.

This is for the NeXus metadata that surrounds the image stream — coordinate axes, detector geometry, and (with dims = &[]) scalar values such as /entry/instrument/detector/distance. Unlike a streaming dataset, it is written once and not appended to.

pub fn write_dataset_typed( &mut self, name: &str, datatype: DatatypeMessage, dims: &[u64], data: &[u8], ) -> Result<usize>

write_dataset with a caller-supplied datatype and the raw element bytes, dims product times the type’s size — for NDFileHDF5’s NUL-terminated fixed-length string constants, where write_string_dataset writes a variable-length one. A byte count that does not match is refused before the dataset is created.

pub fn write_string_dataset( &mut self, name: &str, strings: &[&str], ) -> Result<usize>

Create a variable-length string dataset (one element per string). Returns the dataset index. Useful for NeXus metadata such as /entry/start_time or per-frame timestamp arrays.

The datatype declares UTF-8; write_string_dataset_ascii declares ASCII instead.

pub fn write_string_dataset_ascii( &mut self, name: &str, strings: &[&str], ) -> Result<usize>

write_string_dataset under an ASCII datatype, the one h5py’s string_dtype("ascii") produces. A string that is not 7-bit is rejected rather than mislabelled.

pub fn set_dataset_attr_string( &mut self, ds_index: usize, name: &str, value: &str, ) -> Result<()>

Set a string attribute on a dataset, addressed by its index. The NeXus way to record units, long_name, signal, etc. An existing attribute of the same name is replaced. After start_swmr it is committed at close — see set_group_attr_string.

pub fn set_dataset_attr_numeric<T: H5Type>( &mut self, ds_index: usize, name: &str, value: &T, ) -> Result<()>

Set a numeric scalar attribute on a dataset, addressed by its index. An existing attribute of the same name is replaced. After start_swmr it is committed at close — see set_group_attr_string.

pub fn set_dataset_attr_typed( &mut self, ds_index: usize, name: &str, datatype: DatatypeMessage, value: Vec<u8>, ) -> Result<()>

Set a scalar attribute of a caller-supplied datatype on a dataset, addressed by its index — the SWMR form of H5Dataset::set_attr_typed. After start_swmr it is committed at close — see set_group_attr_string.

pub fn set_dataset_attr_array<T: H5Type>( &mut self, ds_index: usize, name: &str, dims: &[u64], values: &[T], ) -> Result<()>

Set a numeric array attribute on a dataset, addressed by its index.

dims are the dimension sizes (e.g. &[3] for a 1-D array) and the number of values must equal their product. This is the SWMR counterpart of H5Attribute::write_array, for the length-ndims int32 array attributes AreaDetector writes (NDArrayDimOffset, NDArrayDimBinning, NDArrayDimReverse). An existing attribute of the same name is replaced.

After start_swmr it is committed at close — see set_group_attr_string. Resolve a dataset path to its index with dataset_index.

pub fn set_dataset_fill_value<T: H5Type>( &mut self, ds_index: usize, value: &T, ) -> Result<()>

Set the fill value of a streaming dataset, addressed by its index.

Call this before the first append_frame: it determines the value of chunk regions that are never written (a partial final band, or unwritten tiles).

pub fn set_dataset_fill_value_bytes( &mut self, ds_index: usize, bytes: &[u8], ) -> Result<()>

set_dataset_fill_value from one element encoded in the dataset’s stored datatype — for a type no T carries, such as one made with a _with_datatype constructor. bytes is the element’s host-order image, datatype.element_size() bytes long; it is put in the order the datatype declares.

pub fn assign_dataset_to_group( &mut self, group_path: &str, ds_index: usize, ) -> Result<()>

Place an existing dataset inside a group.

By default a dataset created through this writer lives at the root level; this moves its link record into group_path (which must already exist). The group must be created before start_swmr for the placement to be visible to readers during streaming.

pub fn start_swmr(&mut self) -> Result<()>

Signal the start of SWMR mode.

pub fn append_frame(&mut self, ds_index: usize, data: &[u8]) -> Result<()>

Append a frame of raw data to a streaming dataset.

The data size must match one frame (product of frame_dims * element_size).

pub fn append_frame_raw( &mut self, ds_index: usize, data: &[u8], filter_mask: u32, ) -> Result<()>

Append a frame that is already filtered (e.g. compressed by a codec plugin upstream), storing its bytes verbatim — HDF5’s direct chunk write (H5Dwrite_chunk) for a streaming dataset.

data is the stored (compressed) image of one frame; the dataset’s filter pipeline, recorded when it was created with a _compressed constructor, is what readers run to decode it. filter_mask bit i set means filter i was not applied to this frame; pass 0 when the whole pipeline was. The dataset must store one whole frame per chunk (create_streaming_dataset_compressed, or a _chunked_compressed or filtered _with_max_frames chunk of [1, frame_dims...]).

pub fn write_chunk_raw_at( &mut self, ds_index: usize, chunk_coords: &[u64], data: &[u8], filter_mask: u32, ) -> Result<()>

Write one already-filtered chunk verbatim at an explicit chunk position of a grid dataset made by create_grid_dataset_compressed — HDF5’s direct chunk write (H5Dwrite_chunk) at a chunk offset. filter_mask bit i set means filter i was not applied to this chunk; pass 0 when the whole pipeline was.

pub fn write_chunk_at( &mut self, ds_index: usize, chunk_coords: &[u64], data: &[u8], ) -> Result<()>

Write one frame at an explicit chunk position of a grid dataset created with create_grid_dataset.

chunk_coords are in units of chunks (row-major over the chunk grid) and data must be exactly one full chunk (product(chunk) * element_size bytes; edge chunks are zero-padded by the caller). Positions may be written in any order, beyond the extent too while inside the maximum shape, and unwritten positions read back as fill. As with the streaming path, call flush to make writes visible to SWMR readers and set dataset attributes before start_swmr.

pub fn set_extent(&mut self, ds_index: usize, dims: &[u64]) -> Result<()>

Set a dataset’s logical extent within its maximum shape, growing or shrinking it — libhdf5’s H5Dset_extent, e.g. a grid’s extent following the positions written so far. A multi-frame-chunk stream, whose extent counts the frames it buffers, is refused.

pub fn flush(&mut self) -> Result<()>

Flush all dataset index structures to disk with SWMR ordering.

pub fn close(self) -> Result<()>

Close and finalize the file.