neoset 0.1.0

Compact RLE-encoded event sets for efficient peer sync
Documentation
  • Coverage
  • 96.88%
    31 out of 32 items documented1 out of 32 items with examples
  • Size
  • Source code size: 33.2 kB This is the summed size of all the files inside the crates.io package for this release.
  • Documentation size: 3.13 MB This is the summed size of all files generated by rustdoc for all configured targets
  • Ø build duration
  • this release: 11s Average build duration of successful builds.
  • all releases: 11s Average build duration of successful builds in releases after 2024-10-23.
  • Links
  • slightknack/isocore
    1 0 0
  • crates.io
  • Dependencies
  • Versions
  • Owners
  • slightknack

Compact RLE-encoded event sets.

An [EventSet] tracks which events (by index) a peer has seen, using a hybrid run-length / literal encoding for efficient storage across sparse, dense, and random data patterns.

Encoding

Each 64-bit block uses a 2-bit tag in the MSBs:

Tag Meaning Payload
00 Run of N zeros N (62-bit count)
10 Run of N ones N (62-bit count)
D1 Literal 63 raw bits

A literal block packs 63 bits into 64: bit 62 is the literal tag, bit 63 is the MSB of the 63-bit value, and bits 61..0 hold the lower 62 bits. Runs compress homogeneous regions; literals compress mixed/random regions.

Examples

use neoset::{EventSet, EventSetBuilder};

// Dense: "I have all events 0..999"
let dense = EventSet::ones(1000);
assert_eq!(dense.count_ones(), 1000);

// Sparse: "I have events 5, 100, 5000 out of 10000"
let sparse = EventSet::from_ones(&[5, 100, 5000], 10_000);
assert_eq!(sparse.count_ones(), 3);

// Sync diff: what does peer A have that peer B doesn't?
let peer_a = EventSet::ones(1000);
let mut b = EventSetBuilder::new();
b.push_ones(800);
b.push_zeros(50);
b.push_ones(150);
let peer_b = b.finish();
let need: Vec<u64> = peer_a.difference(&peer_b).iter_ones().collect();
assert_eq!(need.len(), 50); // events 800..850