Struct FileAccessProperties 

pub struct FileAccessProperties { /* private fields */ }

File-access properties applied when opening an HDF5 file.

This is the hdf5-pure analogue of an HDF5 file access property list (fapl): one value carrying every access-time setting, built once and passed to whichever open a caller reaches for, exactly as a fapl is handed to H5Fopen. Every *_with_options constructor on File accepts it, so a read path and a read-write path can share one configuration.

The Properties suffix means the type stands in for one whole HDF5 property list, so every setting on it has a C counterpart to look up. It is a stand-in and not a port: a plain Copy value, with no handle to create or close, no runtime property registry, and no setter that can fail. fapl and each H5Pset_* it models are doc aliases, so a search for either lands here.

• The metadata cache (H5Pset_mdc_config) applies to the streaming and bounded backends; an in-memory open already holds the whole file in one buffer.
• The chunk cache (H5Pset_cache) is the file-wide default for datasets opened from any backend, overridable per dataset with DatasetAccessProperties.
• The locking policy (H5Pset_file_locking) applies to the read-write opens. Readers and the SWMR writer take no lock by design, so they ignore it.
• The write-mark policy applies to the read-only opens, and has no C counterpart: H5Fopen refuses a file marked open for write with no override, where with_write_mark_policy can admit a snapshot read of one.

See the property-support reference for the full property-by-property map.

Implementations

impl FileAccessProperties

pub const fn new() -> Self

A value carrying the crate’s default access behavior.

pub const fn with_metadata_cache( self, metadata_cache: MetadataCacheConfig, ) -> Self

Configure the bounded streaming metadata cache.

pub const fn with_chunk_cache(self, chunk_cache: ChunkCacheConfig) -> Self

Configure the per-dataset raw chunk cache used by datasets opened from this file. This is the H5Pset_cache-style file-wide default.

pub const fn with_locking(self, locking: FileLocking) -> Self

Set the OS advisory file-locking policy for the read-write opens.

Defaults to FileLocking::Enabled. Use FileLocking::Disabled only when an external mechanism already guarantees single-writer access, or FileLocking::BestEffort on a filesystem (such as some network mounts) where the OS lock is unavailable. Setting HDF5_USE_FILE_LOCKING in the environment overrides this, as in the C library.

Readers and File::open_swmr_writer take no lock by design and ignore this.

pub const fn with_memory_strategy(self, memory_strategy: MemoryStrategy) -> Self

Set how much memory a read-write open may use to hold the file.

Unset by default, which lets the entry point choose: File::open_rw uses MemoryStrategy::Auto, preferring the bounded engine and falling back to the whole-file mirror for a file it cannot edit. Setting this overrides that default, in either direction, so MemoryStrategy::Bounded refuses such a file rather than quietly spending O(file size) memory on a caller who asked not to; MemoryStrategy::Mirrored takes the whole-file mirror unconditionally, as open_rw did before it learned to dispatch.

The read-only opens ignore this: they build no editing session at all, and their own names say what memory they spend. File::open_swmr_writer does build one, and always mirrors: it accepts MemoryStrategy::Auto and MemoryStrategy::Mirrored, both of which the mirror satisfies, and refuses an explicit MemoryStrategy::Bounded with Error::EditUnsupported rather than quietly not honoring it. Ask a File which backend it resolved to with File::edit_backing.

pub const fn with_libver_bounds(self, low: LibVer, high: LibVer) -> Self

Constrain the on-disk format an editing session may write, mirroring HDF5’s H5Pset_libver_bounds — which the C library classes as a file access property for exactly this reason: it governs what a later write to an existing file is allowed to add.

Unset by default, which keeps File::open_rw adding whatever the content needs. That default is what lets a file the C library wrote under its own bounds be edited at all, but it means a session can add content only a newer library can read without changing the superblock, and the caller has no way to see it happen: adding a chunked, filtered, or resizable dataset to an HDF5 1.8 file needs the version 4 data-layout message and a 1.10 chunk index, since this crate does not write the version 1 B-tree index that 1.8 used.

Setting a high below LibVer::V110 refuses that addition with FormatError::LibverTooOldForContent at File::commit instead — the same refusal FileBuilder::with_libver_bounds gives when writing a whole file, so a .mat bounded to 1.8 for MATLAB stays loadable by MATLAB after an edit.

low only rules formats out — as in the C library it licenses newer encodings without requiring them — so a lower bound of LibVer::V112, LibVer::V114 or LibVer::LATEST leaves the session writing the 1.10 format rather than failing, provided high reaches it. An inverted range such as V114..=V110 is refused with FormatError::LibverBoundsUnsatisfiable.

The read-only opens ignore this: they write nothing. File::open_swmr_writer requires a version 3 superblock, so it refuses a high below LibVer::V110 up front rather than accepting a bound it cannot honor.

pub const fn with_sync_policy(self, sync_policy: SyncPolicy) -> Self

Choose who owns this session’s fsync cadence — this crate, or the application through File::sync.

Defaults to SyncPolicy::Always: every commit and every immediate Dataset::append forces its writes to durable storage before returning. SyncPolicy::OnClose issues no fsync at all, which is what the reference C library does; the writes still reach the operating system by the time the operation making them returns, so only power-loss durability moves to the caller. with_page_buffer_size, off by default, is the one setting that changes that — and it requires this policy.

The read-only opens ignore this: they write nothing.

pub const fn with_page_buffer_size(self, bytes: usize) -> Self

Let a read-write session’s writes accumulate in a page buffer of bytes, so repeated small updates landing in the same page cost one write rather than one each.

Defaults to 0, which is off — as H5Pset_page_buffer_size defaults to off — and leaves the gathering every read-write session already does: one write per dirty page per ordering barrier, so a commit or an append still reaches the operating system in full before it returns. What this buys on top is letting a dirty page survive those barriers, which is where a workload of many small appends into a few pages does most of its repeating. Measured on a paged file, 32 chunk appends into eight datasets followed by a commit: 188 writes with the default gathering and 5 with a page buffer, of which two are the mark below going up and coming down. The appends issue nothing at all until the session ends.

It pays off over a long session, and costs on a short one. The crash mark below is two fsyncs per session whatever the session then does, so there is a break-even: measured on an Apple M1 Max (APFS) with 256-byte appends into eight datasets, 400 appends ran 0.75x — slower — 800 broke even, and 6,400 ran 1.64x. The ratio climbs with session length, because the same pages are re-dirtied more often, and narrows to about 1.1x once 64 KiB payloads rather than metadata churn dominate. One host’s numbers, and the short end is noisy; re-measure on the one that matters with cargo bench --bench hot_paths -- page_buffer, which runs both sides of the crossing.

What it costs, and what pays for it

Gathered writes go out in address order, and every publish point sits below the content it reaches, so all of them are issued first. A write that fails, or a process that dies, mid-flush can therefore leave a file whose superblock, dataset length or object header names bytes that never arrived — and two of those read back clean, as fill values or as a deleted object’s data, with every checksum verifying. That is what a write-back page buffer is, rather than a fault in this one: H5Pset_page_buffer_size reorders the same way and makes no crash-consistency claim either.

So this session raises superblock status-flag bit 0 (H5F_SUPER_WRITE_ACCESS) for its whole life, fsynced once at open and cleared on a clean File::close or drop — the mark the reference C library raises for any writer. A session that dies with pages in memory leaves that byte standing, and a file carrying it is refused by this crate, by H5Fopen and by h5py alike, with Error::FileMarkedInUse. The silent wrong answer becomes a refusal, and File::clear_swmr_flag — the h5clear -s equivalent — is how to look at such a file anyway, knowing what it may hold. A completed commit’s bytes may also still be in this process’s memory when it returns.

Refusals

Four, each refused with Error::EditUnsupported rather than quietly ignored:

• a budget below the page the session merges within: the file’s own file-space page size when it was created with FileSpaceStrategy::Page, and the format’s 4 KiB default otherwise. A buffer that cannot hold one page drains on every page it touches;
• a paged file whose free space is not persisted, which can be neither committed to nor appended to, so the buffer would hold nothing while its mark blocked every reader;
• a superblock older than version 3, whose status-flags byte no library reads back, so the mark above would announce nothing;
• SyncPolicy::Always, the default, where every barrier is an fsync that flushes the buffer on its way out — so it would hold nothing while still costing the mark. Pair this with with_sync_policy(SyncPolicy::OnClose).

File::create_with_options refuses a creation/access pair it could not then reopen with, rather than writing the file first, and File::open_swmr_writer refuses a page buffer outright: its readers observe the order its writes become visible in, which is exactly what a buffer coalesces away.

Choosing a budget

Any budget of at least one page is honored. One below 1 MiB is an explicit request for less resident memory, not a mistake — a writer inside a tight memory cap can ask for 256 KiB and get it — and what it buys that memory with is writes: the budget is the point at which everything held is flushed, so a long contiguous run is flushed and restarted once per budget’s worth of it. Writes issued on a 4 KiB-paged file:

workload	unset	4 KiB	64 KiB	1 MiB
32 chunk appends into 8 datasets, then a commit	188	25	4	4
one 4 MiB append	131	1,094	74	10

On the scattered workload this property exists for, a small budget costs little; on the long run 64 KiB is seven times the writes of 1 MiB.

The memory comparison against leaving this unset is not the one the table suggests. A session that sets nothing already gathers up to 1 MiB of dirty bytes per operation, and releases it at every ordering barrier; a page buffer holds its budget across operations, until the budget is spent, an fsync, or File::close. So 1 MiB here trades a per-operation peak for a continuous residency of the same size, and a budget below 1 MiB lowers both.

How this differs from H5Pset_page_buffer_size

A paged file is not required, where the C library requires one. H5PB_create refuses an unpaged file because the C page buffer is a page cache, and its min_meta_perc / min_raw_perc reservations are counted in pages that the paged allocator keeps segregated by kind. This is a write gatherer: it merges runs within a page-sized window and flushes whole, so a window is all it needs, and an unpaged file gets the same 4 KiB one that every read-write session already gathers under. Since unpaged is the default strategy, requiring Page put this property out of reach of most files for no reason this implementation had.

A small budget costs writes here, where it costs none in C. H5PB_write sends any I/O of a page or more straight to the driver, so a small page_buf_size there caps memory without throttling a long write. Nothing bypasses this buffer — the budget is the point at which everything held is flushed — so a small one turns a single long run into repeated flushes. Both libraries accept the budget; only this one charges for it. See Choosing a budget for what it charges.

A sub-page budget is refused rather than rounded. H5Fopen rounds it up to one page silently, and H5Fcreate refuses it. A property quietly ignored is worse than one refused.

The read-only opens ignore this setting; they write nothing.

Only the budget of H5Pset_page_buffer_size is modeled; its min_meta_perc / min_raw_perc reservations are not, since this buffer does not evict — it flushes whole.

pub const fn with_write_mark_policy(self, policy: WriteMarkPolicy) -> Self

Let a read-only open proceed past a superblock marked open for write by a writer that is not a SWMR writer — status-flag bit 0 alone, which is what with_page_buffer_size raises for a session’s whole life.

Defaults to WriteMarkPolicy::Refuse, which is what H5Fopen does with the same byte: File::open, File::open_streaming and File::from_source all report Error::FileMarkedInUse. WriteMarkPolicy::AllowSnapshot reads the file as it stands instead, through whichever of those opens is passed these properties.

What the caller is asserting

That the writer has flushed: it called File::sync, or it stopped after a flush and the mark stands only because nothing cleared it (a clean File::close takes the mark down, and leaves nothing to opt past). The mark is durable and says nothing about when — a live writer mid-operation and one that exited without closing carry the same byte — so this crate cannot check the assertion, and passing this value is how a caller states it. It is exactly true for a writer under SyncPolicy::OnClose that syncs at the points it wants readable, and it is what the mark exists to guard against when it is false: a page-buffered session’s publish points are written before the content they name, so a snapshot taken mid-flush can show a dataset that reads clean and returns fill values, with every checksum verifying.

The snapshot is of the bytes on disk at open. A buffered open takes it whole; a streaming open reads regions on demand, so a writer that carries on writing can move bytes under it — reach for File::open_with_options when the writer may continue, and for File::open_streaming_with_options when it has stopped and the file is too large to buffer.

What it does not unlock

• a SWMR pair (both bits): that file has a reader of its own, and File::open_swmr follows it — including across the writer’s later appends, which a snapshot cannot;
• File::open_rw and File::open_swmr_writer, which are refused whatever this says. A second writer must not join a file a writer already holds;
• the OS advisory lock, a separate guard with its own policy (with_locking).

A file left marked by a writer that crashed is a different question, and this is not the answer to it: it reads such a file as willingly as a flushed one, and leaves the mark standing for the next reader to meet. File::clear_swmr_flag — the h5clear -s equivalent — is the recovery there, and it records the decision by clearing the byte.

The C library offers no counterpart: H5Fopen refuses the byte with no override, and h5clear is its only way through. This is the narrower one, since it changes nothing on disk.

pub const fn metadata_cache(&self) -> MetadataCacheConfig

Return the configured streaming metadata cache.

pub const fn libver_bounds(&self) -> Option<(LibVer, LibVer)>

Return the configured library-version bounds, or None when an editing session may write whatever its content needs.

pub const fn chunk_cache(&self) -> ChunkCacheConfig

Return the configured per-dataset chunk cache.

pub const fn locking(&self) -> FileLocking

Return the configured file-locking policy.

pub const fn memory_strategy(&self) -> Option<MemoryStrategy>

Return the configured memory strategy, or None when none was asked for and the entry point’s own default applies. This is what was requested; for which backend an open resolved to, see File::edit_backing.

The Option distinguishes “no preference stated” from an explicit MemoryStrategy::Auto, which is what lets an entry point supply its own default without overriding a caller who asked for one; None resolves to MemoryStrategy::Auto, the only default any entry point now supplies rather than as a second break on this accessor.

pub const fn sync_policy(&self) -> SyncPolicy

Return the configured fsync policy.

pub const fn page_buffer_size(&self) -> usize

Return the configured page-buffer budget in bytes; 0 when none was asked for.

pub const fn write_mark_policy(&self) -> WriteMarkPolicy

Return the configured write-mark policy.

Trait Implementations

impl Clone for FileAccessProperties

fn clone(&self) -> FileAccessProperties

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for FileAccessProperties

impl Debug for FileAccessProperties

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for FileAccessProperties

fn default() -> FileAccessProperties

Returns the “default value” for a type.

impl Eq for FileAccessProperties

impl PartialEq for FileAccessProperties

fn eq(&self, other: &FileAccessProperties) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for FileAccessProperties