zblob
Generic, resumable, chunked blob and directory transfer over Zenoh — with BLAKE3 verified streaming (every reply proves itself against a pinnable root before touching disk), range-set resume that survives reconnect and restart, bounded memory, and content-addressed dedup. No application-specific types; it's the large-payload path the Zenoh ecosystem is otherwise missing.
Formerly incubated in the ZenSight monorepo as
zenoh-blob; renamed on graduation — this crate is a community project, not an Eclipse Zenoh deliverable.
Why
Zenoh is excellent at pub/sub and query, but has no turnkey way to move a large
artifact (a debug bundle, a pcap, a dataset, a directory tree) between peers
with progress, integrity, and resume. zblob builds that on the primitives
Zenoh already gives you — multi-reply queryables, a reliable transport, and
CongestionControl::Block backpressure — so you don't fork a file-sync tool to
get it.
The integrity model (wire v3)
A blob's identity is its BLAKE3 bao root. Every transfer chunk travels as a
bao slice: the bytes plus the parent hashes proving them against that root.
Replies are verified as they arrive, out of order, at 16 KiB granularity —
there is no end-of-transfer hash pass, a tampered reply is dropped alone and
re-fetched, and a partial download is always a proven-correct partial. Pin the
root (DownloadRequest::pinned) and a server cannot substitute content at all.
Three tiers
Tier 1 — single blob. One queryable serves every blob under a key prefix.
A download is a manifest GET, then range-set slice GETs
(?ranges=0-5,9,12-20): the client persists a chunk bitfield next to the
.part file and re-queries exactly its holes, so resume, retry, and
arbitrary-hole fetch are the same code path. Memory stays O(chunk_size)
regardless of blob size and arrival order.
Key prefixes are typed by the role they play: a server owns a concrete
ServePrefix, a client asks through a QueryPrefix (which may name several
origins). Serving something implies being able to ask for it, so the
conversion one way is free and the other way is fallible.
use ;
// Server
let serve = new?;
let query = from;
let server = new;
let manifest = server
.register_file
.await?;
let handle = server.spawn.await?; // distribute (id, manifest.root) out of band
// Client — the caller picks the destination; pin the root when you know it.
// A transfer is a call builder: the two things it cannot do without are
// positional, and progress/cancellation/overwrite are set on the builder.
let client = new;
let stats = client
.download_to
.progress
.cancel
.await?;
(This is examples/blob_transfer.rs, which CI compiles and runs.)
Tier 1 also supports push (verified uploads gated by a PushPolicy
authorization hook), availability introspection (…/have bitfields per
responder), replicated servers answering one download cooperatively, and a
filesystem-free path in both directions: download_to_writer fills any
seekable async writer, and upload_source pushes any BlobSource (an
in-memory buffer, a generated artifact) without staging it in a file.
Tier 2 — content-addressed directories (the casync model). A snapshot is
a TreeIndex (a depth-first entry list; files reference their chunks by
content hash) plus a content-addressed chunk store. The client fetches only the
chunks it is missing (needed − have, concurrently, re-hashing each on
receipt) and reconstructs the tree — safely: paths are sanitized, symlinks
materialize last with confined targets, hard links and modes/mtimes round-trip.
Progress is "which hashes are on disk", so an interrupted pull resumes for
free and identical chunks (across files or versions) transfer once. FastCDC
content-defined chunking (seedable gear table) localizes edits so a small
change re-transfers only its neighborhood.
use Arc;
use ;
let store: = new;
let index = build_tree?;
let store_p = new?;
let tree_p = new?;
// Serve it live…
let server = new;
server.register.await?;
let handle = server.spawn.await?;
// …or publish into a router storage, wait for the read-back to settle, and
// exit. See docs/router-storage.md.
new
.snapshots
.coverage
.publish
.await?;
// Consumer — either way, the same call.
let client = new;
client
.download_tree
.await?;
// …or pull one file out of the snapshot without materializing the tree.
let conf = client
.fetch_file
.await?;
(Derived from examples/tree_sync.rs and examples/durable_store.rs.)
Tier 2 also ships a batched fetch (<store>/<algo>/batch — one round for
many chunks instead of one query each), probes that report partial
possession (StoreClient::probe, TreeClient::probe_snapshot) so a client can
pick a holder before fetching, index sharding for snapshots whose index is
itself large, seeding (seed::seed_store satisfies chunks from prior local
copies and zero regions before touching the network), lifecycle
(gc::sweep mark-and-sweep with persistent snapshot tags and in-flight temp
tags), and a DirStore that is atomic, fsynced, optionally zstd-compressed and
optionally sealed with XChaCha20-Poly1305.
Fanout tier — one-to-many rollout (feature fanout). fanout_file
publishes a manifest and its bao slices through a zenoh-ext
AdvancedPublisher, so a late joiner replays the cache and every receiver
verifies each slice against the pinned root exactly as a downloader does. No
resume: an interrupted receiver starts over.
Cargo features
| Feature | What it adds |
|---|---|
zstd |
Per-chunk zstd compression (wire + at-rest), restic-v2 style, with raw bail-out for incompressible data. |
encryption |
XChaCha20-Poly1305 encryption at rest for DirStore (with_encryption), convergent per store key. |
fanout |
One-to-many rollout tier over zenoh-ext AdvancedPublisher (cache + miss detection + late-joiner replay). |
tracing |
tracing events at registration/serve/download/GC points. |
Design notes
- Backpressure is automatic.
Session::getdefaults toCongestionControl::Blockand replies inherit it, so chunk replies block rather than drop under load. The crate sets no congestion control explicitly (the setter is behind Zenoh'sinternalfeature, deliberately not enabled). Reply consolidation is a different knob: clients setConsolidationMode::Noneso replies stream instead of being buffered until query finalization. - Reply keys must match the query. Clients GET the
<prefix>/<id>/**wildcard so theslice/<i>replies are accepted (ReplyKeyExpr::MatchingQuery). - Wire format is postcard with explicit schema-version-first fields and
Zenoh
Encodingtags on every reply; the Tier-2root_hashis a canonical versioned digest (mtime excluded), so byte-identical trees hash identically. - Untrusted input is bounded and validated everywhere: manifest/index sizes, chunk geometry (validated, never clamped), entry paths, symlink targets, allocation caps.
Documentation
The docs/ directory is a full documentation set — start at
docs/README.md:
docs/architecture.md— the three tiers, the shared primitives, and the invariants the design relies on, with diagrams.docs/integrity-model.md,docs/wire-protocol.md,docs/design-decisions.md— the design papers: the BLAKE3/bao core, the keyspace and wire, and the decisions taken and rejected.docs/router-storage.md— run a Zenoh router as the fleet-wide Tier-2 chunk store: serverless transfers, fleet-wide dedup, survival across producer restarts.docs/MIGRATION-v3.md— porting a consumer from wire v2 to v3.CHANGELOG.md— including the full v1 → v2 → v3 migration notes.examples/— runnable end-to-end blob and tree transfers.
Acknowledgements
The design borrows ideas from prior art in the space: iroh-blobs (BLAKE3/bao
verified streaming, range-set requests, tag-based GC), casync and desync
(content-addressed trees, seeding), restic (per-chunk compression container,
seeded chunking), and the FastCDC paper. zblob is an independent
implementation, not a fork of any of them.
License
Licensed under the MIT license.