Module storage 

Recording persistence and reconstruction for scientific state samples.

This module is the complete public storage boundary. Simulations configure named output streams with coordinate-aware sampling intervals through SystemStateWriterBuilder, then offer a borrowed live SystemState to SystemStateWriter::observe_state after each evolution step. The writer checks time before accessing any payload and encodes only streams whose sampling interval includes the current iteration. One bounded queue and worker serve every configured stream. Each stream accumulates an independent byte-targeted chunk entirely in reusable userspace memory and performs filesystem IO only when publishing that chunk. The recording owns exactly one authoritative metadata.json lifecycle.

Ownership and backpressure

Sampling never clones, removes, or retains a scientific payload. The selected values are borrowed only while Serde creates one owned JSONL record. That record is then moved into the recording writer. If the configured queue-byte budget is full, SystemStateWriter::observe_state blocks until the writer commits enough queued bytes or reports a terminal error. Records are never split between chunks.

Lifecycle

SystemStateWriterBuilder::create_new_recording refuses an existing output root, validates every stream against one shared state specification, publishes initial running metadata, and then starts the recording writer. A complete buffered chunk is written once, synchronized, described in metadata, and atomically sealed. SystemStateWriter::complete_recording drains the writer, atomically commits completion timing and terminal metadata, and returns CompletedRecording; SystemStateWriter::mark_recording_failed records an explicit failed lifecycle instead. Dropping an active recording drains its writer thread for memory and file safety but deliberately leaves metadata as running.

SystemStateWriterBuilder::continue_existing_recording explicitly validates and appends an existing running run. SystemStateWriterBuilder::continue_recording_from_latest_checkpoint additionally reconstructs a complete owned checkpoint state through caller-supplied payload decoders. Recovery discards an unpublished temporary chunk, while completing the rename of a descriptor-prepared chunk. SystemStateWriterBuilder::open_or_resume_from_latest_checkpoint is the concise create-or-continue entry point: it infers the newest complete-state checkpoint and removes every stream record later than that checkpoint before writing resumes. Checkpoint-aware continuation verifies the selected latest sealed checkpoint chunk’s exact byte count and SHA-256 checksum before decoding it or returning an append-capable writer.

Reading

StoredStateSeriesReader accepts a completed output directory and a JsonPayloadDecoderRegistry registry. The reader validates metadata, chunks, checksums, record order, and decoder coverage before reconstructing typed StateSeries values. Decoder implementations remain per payload type and registrations remain per exact state key. Latest-state reads verify and decode only the newest chunk.

Boundary

Storage owns durable mechanics: run directories, stream chunking, queue flushing, metadata transitions, and reconstruction integrity checks. Callers own simulation evolution, stream schemas, payload codecs, and scheduling. Storage does not define modeling APIs, RNG behavior, or artifact semantics.

Structs

CompletedRecording

Durable result of a successfully completed recording lifecycle.

CompletedStreamSummary

Aggregate persisted facts for one stream in a completed recording.

JsonPayloadDecoderRegistry

Heterogeneous per-key payload decoder registry.

JsonStringDecoder

Stateless default decoder for payloads stored as String.

JsonVecF64Decoder

Stateless default decoder for payloads stored as Vec<f64>.

RecordingTiming

Immutable operational timing returned after successful recording completion.

StateStreamConfig

Configuration for one independently sampled logical output stream.

StateStreamStorage

Persistence and backpressure policy for one logical state stream.

StoredStateSeriesReader

Reader that reconstructs complete in-memory series from one completed recording.

SystemStateWriter

Exclusive queued writer for all persistent streams in one recording.

SystemStateWriterBuilder

Builder for one exclusive state-recording directory.

TimeAxisMetadata

Public description of the temporal coordinates used by a run.

Enums

SamplingInterval

Coordinate-aware interval used to select states for one output stream.

StateStreamLayout

Filesystem layout for one logical state stream.

StorageError

A failure encountered while encoding, writing, reading, or decoding a run.

Traits

JsonPayloadDecoder

Converts one borrowed raw JSON value into one concrete state payload.