delta-struct 0.3.0

Delta struct provides a rust-lang Deriveable trait, Delta, that can be used to compute the difference (aka delta) between two instances of a type.
Documentation

Compute the difference (delta) between two instances of a type, and apply that difference to a third.

Deriving [Delta] on a struct generates a companion "delta struct" holding only what changed, plus an implementation of the [Delta] trait that knows how to produce one and how to apply it. Pair it with serde and you can send updates over the wire without resending state that both sides already agree on.

Quick start

use delta_struct::Delta;

#[derive(Delta)]
struct Config {
    host: String,
    port: u16,
}

let old = Config { host: "localhost".to_string(), port: 80 };
let new = Config { host: "localhost".to_string(), port: 8080 };

// `Config` gained a companion struct named `ConfigDelta`.
let delta = Delta::delta(old, new).expect("the port changed");
assert_eq!(delta.host, None);          // unchanged fields are `None`
assert_eq!(delta.port, Some(8080));

// Applying the delta to an older copy brings it up to date.
let mut current = Config { host: "localhost".to_string(), port: 80 };
current.apply_delta(delta);
assert_eq!(current.port, 8080);

Note that a single use delta_struct::Delta; imports both the trait and the derive macro. The trait has to be in scope wherever you derive it — the generated code refers to Delta by that name.

[Delta::delta] returns [None] when nothing changed, so if let Some(delta) = Delta::delta(old, new) is the usual way to skip sending an empty update.

Field types

Every field is diffed according to a field type, chosen with #[delta_struct(field_type = "...")]. The default is "scalar", which can be changed per struct — see Container attributes.

scalar (the default)

The field is compared with != and replaced wholesale. In the delta struct it becomes Option<T>: Some(new_value) when the two differ, [None] when they don't. Requires T: PartialEq.

unordered

The field is treated as a bag of elements whose order carries no meaning, so the delta records only which elements came and went: a [BagDelta], holding an add and a remove, both Vec<Item>.

use delta_struct::Delta;
use std::collections::HashSet;

#[derive(Delta)]
struct Device {
    #[delta_struct(field_type = "unordered")]
    services: HashSet<String>,
}

let device = |services: &[&str]| Device {
    services: services.iter().map(|s| s.to_string()).collect(),
};

let delta = Delta::delta(device(&["ssh", "http"]), device(&["http", "mqtt"])).unwrap();
assert_eq!(delta.services.add, vec!["mqtt".to_string()]);
assert_eq!(delta.services.remove, vec!["ssh".to_string()]);

The field has to be a set — a HashSet or a BTreeSet. Formally it needs [Extend] and [TryIndex], this crate's fallible answer to Index; the two std sets implement it, and you can implement it for your own collection. A [Vec] deliberately does not qualify — see Limitations.

Every element of the old collection is looked up in the new one exactly once, so the cost of a diff is the cost of n lookups in whichever collection you picked: O(n) for a HashSet, O(n log n) for a BTreeSet. Applying one costs the same, since each removal is a lookup rather than a rebuild.

apply_delta preserves membership but not position — additions land wherever the collection decides to put them. Use ordered where that matters.

unordered-delta

Like unordered, but for a collection of key/value entries whose values are worth diffing rather than resending. An entry whose key is on both sides is not a removal plus an addition: the two values are handed to [Delta::delta] and only the difference is recorded. The delta is a [MapDelta], holding an add, a remove, and a change.

use delta_struct::Delta;
use std::collections::HashMap;

#[derive(Delta)]
struct Service {
    port: u16,
    healthy: bool,
}

#[derive(Delta)]
struct Cluster {
    #[delta_struct(field_type = "unordered-delta")]
    services: HashMap<String, Service>,
}

let cluster = |port| Cluster {
    services: vec![("web".to_string(), Service { port, healthy: true })]
        .into_iter()
        .collect(),
};

let delta = Delta::delta(cluster(80), cluster(8080)).unwrap();
// `web` stayed put, so all that travels is the one field that moved.
assert!(delta.services.add.is_empty());
assert!(delta.services.remove.is_empty());
assert_eq!(delta.services.change[0].key, "web");
assert_eq!(delta.services.change[0].delta.port, Some(8080));
assert_eq!(delta.services.change[0].delta.healthy, None);

The key is the collection's own — the K of a HashMap<K, V> — not something you nominate. The field has to be a map: a HashMap or a BTreeMap. Formally it needs [Extend] and [TryIndexMut], its entry type needs [MapEntry] (implemented for (K, V), which is what every std map iterates as), and its value type needs [Delta].

[TryIndexMut] rather than [TryIndex] is what excludes sets here, and correctly so: applying a delta means mutating a value where it sits, which a set cannot allow without letting you invalidate the hash or ordering it filed the element under.

Every key of the old collection is looked up in the new one exactly once, so as with unordered the cost is n lookups — O(n) for a HashMap, O(n log n) for a BTreeMap. Applying one preserves membership rather than position, also the same as unordered.

ordered

The field is diffed positionally with Myers' algorithm, and the delta is a minimal edit script: a [SeqDelta] holding [Splice]s that each say "at this index, drop this many items and put these in their place".

use delta_struct::{Delta, Splice};

#[derive(Delta)]
struct Playlist {
    #[delta_struct(field_type = "ordered")]
    tracks: Vec<String>,
}

let old = Playlist { tracks: vec!["intro".to_string(), "b".to_string(), "outro".to_string()] };
let new = Playlist { tracks: vec!["intro".to_string(), "x".to_string(), "outro".to_string()] };

let delta = Delta::delta(old, new).unwrap();
assert_eq!(
    delta.tracks.splices,
    vec![Splice { at: 1, remove: 1, insert: vec!["x".to_string()] }],
);

Splice positions index the old sequence and arrive sorted and non-overlapping, so applying one is a single forward pass. Reordering is a real change here where unordered would see none, and applying a delta reproduces the new sequence exactly, position included.

The collection needs IntoIterator and FromIterator, and its items need Hash + Eq, because that is what indexing the sequences for Myers requires. This is the one field type that takes a [Vec], and so the only one that will diff a sequence at all — but f64 is neither Hash nor Eq, so a Vec<f64> still has nowhere to go but scalar.

delta

The field is itself diffed recursively, which keeps a nested change from resending the whole subtree. Requires the field's type to implement [Delta]; the delta struct holds Option<<T as Delta>::Output>.

use delta_struct::Delta;

#[derive(Delta)]
struct Inner {
    a: i32,
    b: i32,
}

#[derive(Delta)]
struct Outer {
    #[delta_struct(field_type = "delta")]
    inner: Inner,
    name: String,
}

let old = Outer { inner: Inner { a: 1, b: 2 }, name: "x".to_string() };
let new = Outer { inner: Inner { a: 1, b: 3 }, name: "x".to_string() };

let delta = Delta::delta(old, new).unwrap();
let inner_delta = delta.inner.expect("`b` changed");
assert_eq!(inner_delta.a, None);
assert_eq!(inner_delta.b, Some(3));

Container attributes

#[delta_struct(...)] on the struct itself accepts:

  • default = "..." — the field type used for fields without their own field_type. Defaults to "scalar".
  • delta_leader = "..." — tokens to emit immediately above the generated struct. This is how you attach derives, doc comments, or any other attribute to a type you never get to write by hand.
use delta_struct::Delta;
use std::collections::HashSet;

#[derive(Delta)]
#[delta_struct(
    default = "unordered",
    delta_leader = "/// The changes to a `Tags`.\n#[derive(Debug)]"
)]
struct Tags {
    labels: HashSet<String>,
    // Opt an individual field back out of the container default.
    #[delta_struct(field_type = "scalar")]
    revision: u32,
}

let old = Tags { labels: HashSet::new(), revision: 1 };
let new = Tags {
    labels: vec!["new".to_string()].into_iter().collect(),
    revision: 2,
};
let delta = Delta::delta(old, new).unwrap();
assert_eq!(format!("{:?}", delta.labels.add), r#"["new"]"#);
assert_eq!(delta.revision, Some(2));

delta_leader also works on individual fields, where it decorates the generated field instead of the generated struct.

# use delta_struct::Delta;
#[derive(Delta)]
struct Host {
    #[delta_struct(delta_leader = "/// The new port, if it moved.")]
    port: u16,
}

Working with serde

For scalar and delta fields there is no serde integration to enable; delta_leader is the whole story. Put the derives on the generated struct and it serializes like anything else:

use delta_struct::Delta;

#[derive(Delta)]
#[delta_struct(delta_leader = "#[derive(serde::Serialize, serde::Deserialize)]")]
struct Config {
    host: String,
    port: u16,
}

let old = Config { host: "localhost".to_string(), port: 80 };
let new = Config { host: "localhost".to_string(), port: 8080 };

// Sender: there is no message to send at all when nothing changed.
let payload = Delta::delta(old, new).map(|delta| serde_json::to_string(&delta).unwrap());
assert_eq!(payload.as_deref(), Some(r#"{"host":null,"port":8080}"#));

// Receiver applies it to whatever it already had.
let mut config = Config { host: "localhost".to_string(), port: 80 };
config.apply_delta(serde_json::from_str::<ConfigDelta>(&payload.unwrap()).unwrap());
assert_eq!(config.port, 8080);

Field-level delta_leader carries serde attributes just as well, so skip_serializing_if can keep unchanged fields out of the payload entirely rather than sending them as null:

use delta_struct::Delta;

#[derive(Delta)]
#[delta_struct(delta_leader = "#[derive(serde::Serialize)]")]
struct Config {
    #[delta_struct(delta_leader = "#[serde(skip_serializing_if = \"Option::is_none\")]")]
    host: String,
    #[delta_struct(delta_leader = "#[serde(skip_serializing_if = \"Option::is_none\")]")]
    port: u16,
}

let old = Config { host: "localhost".to_string(), port: 80 };
let new = Config { host: "localhost".to_string(), port: 8080 };

let delta = Delta::delta(old, new).unwrap();
assert_eq!(serde_json::to_string(&delta).unwrap(), r#"{"port":8080}"#);

Checking that a delta belongs

[Delta::apply_delta] assumes the value it is handed equals the old the delta came from, and checks nothing. Over an unreliable transport that assumption breaks: a message is dropped, delivered twice, or arrives at a receiver whose state drifted for some other reason, and the two sides diverge in silence.

[Versioned] is the opt-in fix. It pairs a value with a version counter and a [Fingerprint] of its contents, and refuses any delta that does not belong.

use delta_struct::{Applied, Delta, Fingerprint, Mismatch, Versioned};

#[derive(Clone, Debug, Delta, Fingerprint, PartialEq)]
#[delta_struct(delta_leader = "#[derive(Clone)]")]
struct Config {
    host: String,
    port: u16,
}

let config = |port| Config { host: "localhost".to_string(), port };

let mut sender = Versioned::new(config(80));
let mut receiver = Versioned::new(config(80));

let first = sender.commit(config(8080)).expect("the port changed");
let second = sender.commit(config(9090)).expect("the port changed again");

// Delivered twice: recognised and ignored.
assert_eq!(receiver.apply(first.clone()), Ok(Applied::Updated));
assert_eq!(receiver.apply(first), Ok(Applied::Stale));
assert_eq!(receiver.apply(second), Ok(Applied::Updated));
assert_eq!(receiver.get(), sender.get());

// A delta from a stream this receiver never joined is refused rather than
// half-applied.
let mut stranger = Versioned::new(config(80));
let orphan = Versioned::new(config(1)).commit(config(2)).unwrap();
assert!(matches!(stranger.apply(orphan), Err(Mismatch::Base { .. })));

Every [VersionedDelta] carries four numbers, each catching a failure the others cannot:

Field Catches
from, to A message dropped, reordered, or replayed.
base A receiver whose state drifted for any reason, including one that never came through this stream.
result The delta itself being wrong — mismatched schema versions, or a bug.

A rejected delta leaves the receiver untouched and its version unmoved, so a later delta in the same stream fails too rather than papering over the hole. The answer to any [Mismatch] is to resend the whole [Versioned], which serializes as a unit and carries the version the receiver resumes from.

None of this touches the [Delta] trait, the derive, or any generated struct. If you are diffing locally rather than over a wire, you never name anything in this section and pay for none of it.

Fingerprint

[Fingerprint] is a separate derive because [std::hash::Hash] cannot do the job: it is not implemented for HashSet or HashMap — exactly the collections the unordered field types require — and its standard hasher is allowed to change between Rust releases, which would make a toolchain upgrade on one side of a connection look like corruption.

So sets and maps fold commutatively, iteration order cannot reach the result, and the hash is pinned to FNV-1a constants written down in the source. The same value fingerprints identically on any platform and any Rust version. Unlike [Delta], it derives on enums too.

Checking costs a full traversal of the state on each commit and each apply — cheaper than serializing it, but not free, which is the price of the base and result guarantees.

What gets generated

For struct Foo, deriving [Delta] emits struct FooDelta with the same visibility as Foo and the same generic parameters, carrying over their bounds and where clause as written. All of its fields are pub, and by default it derives nothing at all — reach for delta_leader whenever you need Debug, Clone, or serde on it. (Likewise if your crate sets #![deny(missing_docs)]: the generated struct and its fields need doc comments supplied through delta_leader.)

Every field type maps one source field onto exactly one delta field, so a delta struct always has the same fields in the same order as the struct it came from — only their types differ. A tuple struct's delta is a tuple struct in turn, so its fields keep their positions:

use delta_struct::Delta;

#[derive(Delta)]
struct Meters(i32);

let delta = Delta::delta(Meters(3), Meters(4)).unwrap();
assert_eq!(delta.0, Some(4));

Limitations

  • Structs only. Enums and unions are rejected; there is no obvious delta for a value that changed variant.
  • Every type parameter gets a PartialEq bound on the generated impl, whether or not the field that uses it needs one.
  • A unit struct's delta is always [None], as is that of a struct with no fields — there is nothing that could differ.
  • ordered items need Hash + Eq, so float sequences are out. See that section above.
  • A [Vec] cannot be an unordered field. Membership diffing goes through [TryIndex], and a Vec has no sub-linear lookup to offer — implementing it would only hide a quadratic scan behind an O(1)-looking call. Use a HashSet or a BTreeSet, or ordered if position matters.
  • unordered-delta keys are the collection's own. There is no way to nominate a field of the value as the key, so a Vec<Record> has to become a HashMap<Id, Record> to use it.
  • [Versioned] assumes one writer per stream. Two senders committing against the same base both produce from: 0, and the second is rejected rather than merged. Divergence is detected, not reconciled — reach for a CRDT if you need concurrent writers.