eth-state-diff 0.1.1

Minimal-allocation delta encoding for Ethereum consensus state, optimized for archival storage and zstd compression.
Documentation
  • Coverage
  • 69.88%
    181 out of 259 items documented0 out of 68 items with examples
  • Size
  • Source code size: 80.15 kB This is the summed size of all the files inside the crates.io package for this release.
  • Documentation size: 4.26 MB This is the summed size of all files generated by rustdoc for all configured targets
  • Ø build duration
  • this release: 13s Average build duration of successful builds.
  • all releases: 15s Average build duration of successful builds in releases after 2024-10-23.
  • Links
  • galadd/eth_state_diff
    1 0 0
  • crates.io
  • Dependencies
  • Versions
  • Owners
  • galadd

eth_state_diff

Crates.io Docs.rs

eth_state_diff computes and applies compact binary deltas between Ethereum consensus-layer states. It is designed for archival storage, fast-sync, and historical state reconstruction, replacing full snapshots with highly compressible encodings.

This library targets the disk layer — the output is always a compact, serializable delta.

Consensus clients using persistent or tree-backed data structures will need to materialize these flat views before calling create. While this materialization introduces overhead to the integration step, it allows the diffing engine itself to operate at maximum memory bandwidth, strictly avoiding the cache-thrashing penalties of per-element tree traversals on dense data.

Note on Integration Overhead: The primary bottleneck in adopting this library for tree-backed clients is the flattening step. Future optimizations may explore streaming SSZ serializers or tree-specific traversal helpers, and contributions in this area are highly welcome.

Features

  • Domain-specific encoders: Mode-corrected zigzag varints for dense balance updates, sparse delta-varints for participation flags, and field-level SSZ patches for validators.
  • Hardware-optimized: Linear scans over flat arrays designed to maximize CPU prefetching and memory bus utilization.
  • Zero-copy deserialization: Native rkyv support for instant access to archived delta data without allocation.
  • Entropy-optimal: Output payloads are structured to compress to extreme degrees using standard tools like zstd (e.g., ~900KB for 32 epochs of mainnet state).
  • Minimal allocations during apply: Reconstruction modifies target buffers in-place, requiring heap allocations only for newly appended validators.
  • Generic integration: Simple traits (DiffSource and DiffTarget) allow integration with any consensus client that can provide or consume raw SSZ byte slices.

License

Licensed under either of

at your option.