Skip to main content

Module stream

Module stream 

Source
Expand description

Object/manifest-level, content-addressed streaming over a Store.

StreamStore is the foundation for store-to-store sync: it exposes the raw, content-addressed blob and manifest primitives an orchestrator needs to copy a snapshot directly from one store to another — through memory, with no local filesystem staging.

Where Store works at the snapshot level (read a whole tree out with get_manifest + fetch_files, write one in with push), StreamStore works at the object level: check whether a single content object is already present (has_object), read one raw blob by its content-address (get_object), write one raw blob (put_object), and write the manifest object itself (put_manifest). A later orchestrator can then walk a source manifest, get_object each referenced blob from the source, put_object it into the destination (skipping any the destination already has_object), and finally put_manifest — never touching the local disk.

Every read and write is BLAKE3-verified against the address it is filed under (the same integrity discipline as Store): a blob whose bytes do not hash to its checksum is rejected with StoreError::Integrity rather than returned or stored, so corruption can never silently propagate across a store-to-store copy.

The sharded object/manifest keys and the manifest byte-format are reused verbatim from each backend’s existing Store implementation, so a StreamStore round-trip is byte-for-byte interchangeable with a push / fetch_files round-trip (and with the Bash oracle’s layout).

Like Store, the trait is synchronous: the network backends drive their async SDK calls on an internal runtime via block_on, exactly as their Store methods do. It is not implemented for the external-store shim (ExternalStore), which is shell- and local-path-based and cannot stream raw object blobs.

Traits§

StreamStore
Raw, content-addressed object/manifest streaming on top of a Store.