Expand description
Streaming store-to-store snapshot copy.
sync_snapshot copies ONE snapshot — its manifest plus every raw object
it references — directly from a source StreamStore to a destination
StreamStore, through memory only. There is no local filesystem
staging: the function signature deliberately takes no
Path anywhere, so a blob read out of the source can
only ever flow into the destination (it never touches scratch/cache on disk).
§Sync methods, rayon threads — not the async driver
StreamStore’s methods are synchronous: the network backends drive
their async SDK calls on an internal runtime via block_on. Driving them
from the async run_concurrent /
RateLimiter would nest one tokio runtime
inside another and panic. So this orchestrator parallelizes object copies
across a rayon ThreadPool sized to
TransferConfig::concurrency — exactly the pattern
FileStore::parallel_copy. Rayon workers are
plain OS threads, so each one may safely call the block_on-ing sync
get_object/put_object. Bandwidth is throttled by the synchronous
BlockingRateLimiter (one shared bucket via Arc), never the async
RateLimiter.
§Invariants
- Skip-present / incremental: an object the destination already
has_objectis not re-copied. - Manifest-last / all-or-nothing: the destination manifest is written
only after every referenced object has landed. On the first object error
the copy stops and NO manifest is written, so a destination manifest always
implies its objects are present (mirroring
push). - Verified: every blob is BLAKE3-verified by the underlying
StreamStoreon both read and write, and the source manifest is verified to hash toidbyget_manifest.
Structs§
- Sync
Report - Outcome of a
sync_snapshotcall.
Functions§
- sync_
snapshot - Copies one snapshot’s manifest + raw objects directly from
fromtoto, through memory only (no local filesystem staging).