Skip to main content

Module wal_export

arkhe_forge_platform

Module wal_export 

Source
Expand description

WAL streaming export — incremental record append for snapshot / backup.

§Three firm requirements

  1. Fixed-width u64 big-endian length prefix before every record in the stream — deterministic + cross-platform; varint routes through a separate determinism verify path.
  2. Bounds-check on the length prefix before any deref of record bytes — 0 < prefix ≤ MAX_RECORD_BYTES. Malformed prefixes MUST be rejected with WalExportError::InvalidFraming(InvalidFramingReason). Pattern mirrors the host-fn (ptr, len) bounds-check contract.
  3. Stream header magic STREAM_HEADER_MAGIC (ARKHEXP1) pinned at the beginning of every export stream so truncation can be distinguished from “the consumer is reading garbage that happens to start with a valid length prefix”.

§Type-level append-only invariant (A14 projection)

WalRecordSink exposes only WalRecordSink::append_record and WalRecordSink::flush. Seek / truncate / rewrite operations are deliberately absent from the trait surface — external callers cannot invoke them through the trait, and the L0 A14 append-only invariant is projected into the L2 export layer without touching L0 source (DO NOT TOUCH #7 preserved). Concrete implementations MAY add private internal positioning helpers but MUST NOT expose them on the public surface.

§DO NOT TOUCH #7 (postcard field order) invariant

The streaming format wraps unmodified L0 record bytes. The length-prefix + stream-header framing wraps the protected payload from the outside; postcard field order inside each record is preserved bit-exact. The wire-stability tests verify this by comparing streamed record bytes against arkhe_kernel::persist::Wal::serialize record bytes.

§Forward-compatibility

All public types are #[non_exhaustive] — adding variants or fields does not break external matchers. WalRecordSink trait may grow additional methods only if they preserve the append-only invariant (no seek / truncate / rewrite).

§Stability

Wire-format surface frozen. Subsequent magic variants are added through migration mechanisms, never silent reuse — see StreamMagic::recognize for the reader-side dispatch.

Re-exports§

pub use buffered_sink::BufferedWalSink;
pub use reader::StreamingWalReader;
pub use reader::WalStreamReader;

Modules§

buffered_sink
BufferedWalSink<W> — concrete WAL streaming sink.
reader
WalStreamReader — production reader-side surface.

Enums§

InvalidFramingReason
Reasons a streamed framing wrapper is rejected.
StreamMagic
Stream-header magic version dispatch — cross-version forward-compat dispatch surface. Each enum variant maps to a distinct 8-byte ASCII tag at the start of an exported stream; the reader dispatches on the recognised tag to select the corresponding frame-format parser.
WalExportError
Top-level streaming export error surface.

Constants§

MAX_RECORD_BYTES
Maximum byte length accepted in a single record’s length prefix.
STREAM_HEADER_MAGIC
Magic bytes pinned at the start of every WAL export stream.

Traits§

WalRecordSink
Append-only sink for streaming WAL records.