DeltaStream
DeltaStream turns serializable Rust state into validated snapshot or delta packets that can travel over any byte-capable transport.
State → Publisher → bytes → transport → bytes → Subscriber → synchronized state
It works above TCP, WebSocket, PubNub, MQTT, NATS, IPC, files, or any custom transport.
Latest release:
0.32.0
Status: Public preview
Why DeltaStream?
Sending a complete state object after every small update wastes bandwidth and makes recovery logic your problem.
DeltaStream provides:
- snapshot and delta generation;
- automatic snapshot-versus-delta selection;
- optional zstd compression;
- sequence and duplicate detection;
- state-hash validation;
- CRC32 packet integrity checks;
- explicit recovery after packet loss;
- bounded decoding for untrusted input;
- transport-independent serialized bytes.
Installation
[]
= "0.31.0"
= { = "1", = ["derive"] }
Quick Start
use ;
use ;
The common path is intentionally small:
let bytes = publisher.encode?;
subscriber.receive?;
Updates
The first call normally creates a snapshot. Later compatible changes may be encoded as smaller deltas.
let first = PlayerState ;
subscriber.receive?;
let updated = PlayerState ;
subscriber.receive?;
assert_eq!;
Packet-Loss Recovery
When a required delta is missing, the subscriber returns Apply::NeedSnapshot.
use Apply;
match subscriber.receive?
Create a recovery snapshot with:
let packet = publisher.recovery_snapshot?;
subscriber.apply?;
Recovery snapshots preserve the publisher’s current sequence, so repairing one subscriber does not disrupt healthy subscribers.
Lower-Level Packet API
Use the packet API when you need metadata inspection, persistence, custom buffering, or transport-specific control.
use Packet;
let packet = publisher.update?;
let bytes = packet.to_bytes?;
let packet = from_bytes?;
let result = subscriber.apply?;
The high-level methods are equivalent to:
Publisher::encode = Publisher::update + Packet::to_bytes
Subscriber::receive = Packet::from_bytes + Subscriber::apply
Decode Safety
Packet::from_bytes and Subscriber::receive use bounded defaults:
- maximum encoded packet size:
64 MiB + header; - maximum decompressed payload:
64 MiB.
Use DecodeConfig for tighter deployment limits:
use ;
let config = DecodeConfig ;
let subscriber = with_decode_config;
Malformed, truncated, corrupted, unsupported, or oversized packets return controlled errors instead of panicking.
A rejected packet does not partially mutate subscriber state.
Compatibility
DeltaStream 0.31.0 writes wire version 3.
The current decoder accepts:
wire versions 2..=3
Schema compatibility is based on the DeltaState schema hash. Incompatible deltas are rejected, while snapshot migrations remain explicit through MigrationRegistry.
Optional CRDT Support
The crdt feature adds a separate multi-writer replicated-value layer. It does not change the authoritative Publisher/Subscriber synchronization API.
[]
= { = "0.31.0", = ["crdt"] }
Current CRDT types are GCounter, PNCounter, and LwwRegister. Their state-based merge operations tolerate duplicate and reordered delivery after every replica eventually receives the same states. The transport still handles delivery, authentication, encryption, and persistence.
use ;
#
LwwRegister timestamps are supplied by the application; use monotonic logical timestamps, Lamport clocks, or another suitable ordering source. This feature does not claim arbitrary Rust struct CRDT support or collaborative-document semantics. See docs/CRDT.md.
Feature Flags
| Feature | Purpose |
|---|---|
derive |
Re-export #[derive(DeltaState)]. Enabled by default. |
zstd-compression |
Enable adaptive zstd candidates. Enabled by default. |
crdt |
Enable optional state-based CRDT types. Disabled by default. |
pubnub-transport |
Enable the PubNub adapter. |
websocket-transport |
Enable the WebSocket adapter. |
mqtt-transport |
Enable the MQTT adapter. |
nats-transport |
Enable the NATS adapter. |
all-transports |
Enable every transport adapter. |
full |
Enable derive, compression, and all transports. |
Transport features remain optional, so the core crate does not require an async runtime unless a selected adapter needs one.
Examples
cargo run --example basic_sync
cargo run --example recovery
basic_syncdemonstrates snapshot and delta synchronization.recoverydrops a packet, detects the gap, applies a recovery snapshot, and verifies convergence.
Validation
cargo fmt --all -- --check
cargo check --workspace --all-features
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo test --workspace --all-features
cargo bench --no-run
Run heavy ignored tests explicitly:
cargo test --workspace --all-features -- --ignored --nocapture
Fuzzing
The repository includes fuzz targets for packet decoding and subscriber receiving.
rustup toolchain install nightly
cargo install cargo-fuzz
cargo +nightly fuzz run packet_from_bytes
cargo +nightly fuzz run subscriber_receive
Native Windows fuzzing may require the Visual Studio C++ AddressSanitizer runtime.
Both targets completed extended pre-release runs for 0.31.0 without crash artifacts or sanitizer failures. Fuzzing does not replace a formal security audit.
Limitations
DeltaStream does not provide:
- network delivery guarantees;
- authentication or authorization;
- encryption;
- global persistence;
- cross-language protocol stability;
- automatic arbitrary schema migration;
- media compression.
High-entropy or completely unrelated state changes may be more efficient as snapshots.
Before 1.0, minor releases may refine APIs and compatibility rules.
Documentation
- Architecture
- Protocol and recovery
- Compatibility
- Optional CRDT support
- Benchmarks
- Security
- Release process
- Changelog
License
Licensed under the MIT License.