Compute the difference (delta) between two instances of a type, and apply that difference to a third.
Deriving [Delta] on a struct generates a companion "delta struct" holding
only what changed, plus an implementation of the [Delta] trait that knows
how to produce one and how to apply it. Pair it with serde and you can
send updates over the wire without resending state that both sides already
agree on.
Quick start
use Delta;
let old = Config ;
let new = Config ;
// `Config` gained a companion struct named `ConfigDelta`.
let delta = delta.expect;
assert_eq!; // unchanged fields are `None`
assert_eq!;
// Applying the delta to an older copy brings it up to date.
let mut current = Config ;
current.apply_delta;
assert_eq!;
Note that a single use delta_struct::Delta; imports both the trait and
the derive macro. The trait has to be in scope wherever you derive it — the
generated code refers to Delta by that name.
[Delta::delta] returns [None] when nothing changed, so
if let Some(delta) = Delta::delta(old, new) is the usual way to skip
sending an empty update.
Field types
Every field is diffed according to a field type, chosen with
#[delta_struct(field_type = "...")]. The default is "scalar", which can
be changed per struct — see Container attributes.
scalar (the default)
The field is compared with != and replaced wholesale. In the delta struct
it becomes Option<T>: Some(new_value) when the two differ, [None]
when they don't. Requires T: PartialEq.
unordered
The field is treated as a bag of elements whose order carries no meaning,
so the delta records only which elements came and went: a [BagDelta],
holding an add and a remove, both Vec<Item>.
use Delta;
use HashSet;
let device = ;
let delta = delta.unwrap;
assert_eq!;
assert_eq!;
The field has to be a set — a HashSet or
a BTreeSet. Formally it needs [Extend]
and [TryIndex], this crate's fallible answer to
Index; the two std sets implement it, and you can
implement it for your own collection. A [Vec] deliberately does not
qualify — see Limitations.
Every element of the old collection is looked up in the new one exactly
once, so the cost of a diff is the cost of n lookups in whichever
collection you picked: O(n) for a HashSet, O(n log n) for a
BTreeSet. Applying one costs the same, since each removal is a lookup
rather than a rebuild.
apply_delta preserves membership but not position — additions land
wherever the collection decides to put them. Use ordered where that
matters.
unordered-delta
Like unordered, but for a collection of key/value entries whose values
are worth diffing rather than resending. An entry whose key is on both
sides is not a removal plus an addition: the two values are handed to
[Delta::delta] and only the difference is recorded. The delta is a
[MapDelta], holding an add, a remove, and a change.
use Delta;
use HashMap;
let cluster = ;
let delta = delta.unwrap;
// `web` stayed put, so all that travels is the one field that moved.
assert!;
assert!;
assert_eq!;
assert_eq!;
assert_eq!;
The key is the collection's own — the K of a HashMap<K, V> — not
something you nominate. The field has to be a map: a
HashMap or a
BTreeMap. Formally it needs [Extend] and
[TryIndexMut], its entry type needs [MapEntry] (implemented for
(K, V), which is what every std map iterates as), and its value type
needs [Delta].
[TryIndexMut] rather than [TryIndex] is what excludes sets here, and
correctly so: applying a delta means mutating a value where it sits, which
a set cannot allow without letting you invalidate the hash or ordering it
filed the element under.
Every key of the old collection is looked up in the new one exactly once,
so as with unordered the cost is n lookups — O(n) for a HashMap,
O(n log n) for a BTreeMap. Applying one preserves membership rather than
position, also the same as unordered.
ordered
The field is diffed positionally with Myers' algorithm, and the delta is a
minimal edit script: a [SeqDelta] holding [Splice]s that each say
"at this index, drop this many items and put these in their place".
use ;
let old = Playlist ;
let new = Playlist ;
let delta = delta.unwrap;
assert_eq!;
Splice positions index the old sequence and arrive sorted and
non-overlapping, so applying one is a single forward pass. Reordering is a
real change here where unordered would see none, and applying a delta
reproduces the new sequence exactly, position included.
The collection needs IntoIterator and FromIterator, and its items need
Hash + Eq, because that is what indexing the sequences for Myers
requires. This is the one field type that takes a [Vec], and so the only
one that will diff a sequence at all — but f64 is neither Hash nor
Eq, so a Vec<f64> still has nowhere to go but scalar.
delta
The field is itself diffed recursively, which keeps a nested change from
resending the whole subtree. Requires the field's type to implement
[Delta]; the delta struct holds Option<<T as Delta>::Output>.
use Delta;
let old = Outer ;
let new = Outer ;
let delta = delta.unwrap;
let inner_delta = delta.inner.expect;
assert_eq!;
assert_eq!;
Container attributes
#[delta_struct(...)] on the struct itself accepts:
default = "..."— the field type used for fields without their ownfield_type. Defaults to"scalar".delta_leader = "..."— tokens to emit immediately above the generated struct. This is how you attach derives, doc comments, or any other attribute to a type you never get to write by hand.
use Delta;
use HashSet;
let old = Tags ;
let new = Tags ;
let delta = delta.unwrap;
assert_eq!;
assert_eq!;
delta_leader also works on individual fields, where it decorates the
generated field instead of the generated struct.
# use Delta;
Working with serde
For scalar and delta fields there is no serde integration to enable;
delta_leader is the whole story. Put the derives on the generated struct
and it serializes like anything else:
use Delta;
let old = Config ;
let new = Config ;
// Sender: there is no message to send at all when nothing changed.
let payload = delta.map;
assert_eq!;
// Receiver applies it to whatever it already had.
let mut config = Config ;
config.apply_delta;
assert_eq!;
Field-level delta_leader carries serde attributes just as well, so
skip_serializing_if can keep unchanged fields out of the payload
entirely rather than sending them as null:
use Delta;
let old = Config ;
let new = Config ;
let delta = delta.unwrap;
assert_eq!;
Checking that a delta belongs
[Delta::apply_delta] assumes the value it is handed equals the old the
delta came from, and checks nothing. Over an unreliable transport that assumption breaks:
a message is dropped, delivered twice, or arrives at a receiver whose state
drifted for some other reason, and the two sides diverge in silence.
[Versioned] is the opt-in fix. It pairs a value with a version counter
and a [Fingerprint] of its contents, and refuses any delta that does not
belong.
use ;
let config = ;
let mut sender = new;
let mut receiver = new;
let first = sender.commit.expect;
let second = sender.commit.expect;
// Delivered twice: recognised and ignored.
assert_eq!;
assert_eq!;
assert_eq!;
assert_eq!;
// A delta from a stream this receiver never joined is refused rather than
// half-applied.
let mut stranger = new;
let orphan = new.commit.unwrap;
assert!;
Every [VersionedDelta] carries four numbers, each catching a failure the
others cannot:
| Field | Catches |
|---|---|
from, to |
A message dropped, reordered, or replayed. |
base |
A receiver whose state drifted for any reason, including one that never came through this stream. |
result |
The delta itself being wrong — mismatched schema versions, or a bug. |
A rejected delta leaves the receiver untouched and its version unmoved, so
a later delta in the same stream fails too rather than papering over the
hole. The answer to any [Mismatch] is to resend the whole [Versioned],
which serializes as a unit and carries the version the receiver resumes
from.
None of this touches the [Delta] trait, the derive, or any generated
struct. If you are diffing locally rather than over a wire, you never name
anything in this section and pay for none of it.
Fingerprint
[Fingerprint] is a separate derive because [std::hash::Hash] cannot do
the job: it is not implemented for HashSet
or HashMap — exactly the collections the
unordered field types require — and its standard hasher is allowed to
change between Rust releases, which would make a toolchain upgrade on one
side of a connection look like corruption.
So sets and maps fold commutatively, iteration order cannot reach the
result, and the hash is pinned to FNV-1a constants written down in the
source. The same value fingerprints identically on any platform and any
Rust version. Unlike [Delta], it derives on enums too.
Checking costs a full traversal of the state on each commit and each
apply — cheaper than serializing it, but not free, which is the price of
the base and result guarantees.
What gets generated
For struct Foo, deriving [Delta] emits struct FooDelta with the same
visibility as Foo and the same generic parameters, carrying over their
bounds and where clause as written. All of its fields are
pub, and by default it derives nothing at all — reach for delta_leader
whenever you need Debug, Clone, or serde on it. (Likewise if your crate
sets #![deny(missing_docs)]: the generated struct and its fields need doc
comments supplied through delta_leader.)
Every field type maps one source field onto exactly one delta field, so a delta struct always has the same fields in the same order as the struct it came from — only their types differ. A tuple struct's delta is a tuple struct in turn, so its fields keep their positions:
use Delta;
;
let delta = delta.unwrap;
assert_eq!;
Limitations
- Structs only. Enums and unions are rejected; there is no obvious delta for a value that changed variant.
- Every type parameter gets a
PartialEqbound on the generated impl, whether or not the field that uses it needs one. - A unit struct's delta is always [
None], as is that of a struct with no fields — there is nothing that could differ. ordereditems needHash + Eq, so float sequences are out. See that section above.- A [
Vec] cannot be anunorderedfield. Membership diffing goes through [TryIndex], and aVechas no sub-linear lookup to offer — implementing it would only hide a quadratic scan behind an O(1)-looking call. Use aHashSetor aBTreeSet, ororderedif position matters. unordered-deltakeys are the collection's own. There is no way to nominate a field of the value as the key, so aVec<Record>has to become aHashMap<Id, Record>to use it.- [
Versioned] assumes one writer per stream. Two senders committing against the same base both producefrom: 0, and the second is rejected rather than merged. Divergence is detected, not reconciled — reach for a CRDT if you need concurrent writers.