Skip to main content

Crate delta_struct

Crate delta_struct 

Source
Expand description

Compute the difference (delta) between two instances of a type, and apply that difference to a third.

Deriving Delta on a struct generates a companion “delta struct” holding only what changed, plus an implementation of the Delta trait that knows how to produce one and how to apply it. Pair it with serde and you can send updates over the wire without resending state that both sides already agree on.

§Quick start

use delta_struct::{Delta, ScalarDelta};

#[derive(Delta)]
struct Config {
    host: String,
    port: u16,
}

let old = Config { host: "localhost".to_string(), port: 80 };
let new = Config { host: "localhost".to_string(), port: 8080 };

// `Config` gained a companion struct named `ConfigDelta`.
let delta = Delta::delta(old, new).expect("the port changed");
assert_eq!(delta.host, ScalarDelta::Unchanged);
assert_eq!(delta.port, ScalarDelta::Changed(8080));

// Applying the delta to an older copy brings it up to date.
let mut current = Config { host: "localhost".to_string(), port: 80 };
current.apply_delta(delta).unwrap();
assert_eq!(current.port, 8080);

Note that a single use delta_struct::Delta; imports both the trait and the derive macro. The trait has to be in scope wherever you derive it — the generated code refers to Delta by that name.

Delta::delta returns None when nothing changed, so if let Some(delta) = Delta::delta(old, new) is the usual way to skip sending an empty update.

Delta::apply_delta returns a Result, and the unwrap above is safe rather than lazy: a struct’s delta always fits the struct, so only an enum can fail — see Mismatch. Use ? wherever an enum is in reach.

§Field types

Every field is diffed according to a field type, chosen with #[delta_struct(field_type = "...")]. The default is "scalar", which can be changed per struct — see Container attributes.

§scalar (the default)

The field is compared with != and replaced wholesale. In the delta struct it becomes ScalarDelta<T>: ScalarDelta::Changed(new_value) when the two differ, ScalarDelta::Unchanged when they don’t. Requires T: PartialEq.

§unordered

The field is treated as a collection whose order carries no meaning, so the delta records only which elements came and went. A set’s answer to that is a BagDelta, holding an add and a remove, both Vec<Item>.

use delta_struct::Delta;
use std::collections::HashSet;

#[derive(Delta)]
struct Device {
    #[delta_struct(field_type = "unordered")]
    services: HashSet<String>,
}

let device = |services: &[&str]| Device {
    services: services.iter().map(|s| s.to_string()).collect(),
};

let delta = Delta::delta(device(&["ssh", "http"]), device(&["http", "mqtt"])).unwrap();
assert_eq!(delta.services.add, vec!["mqtt".to_string()]);
assert_eq!(delta.services.remove, vec!["ssh".to_string()]);

The field has to be a set or a map — a HashSet, a BTreeSet, a HashMap, or a BTreeMap. Formally it needs Unordered, which all four implement and which you can implement for your own collection. A Vec deliberately does not qualify — see Limitations.

Every element of the old collection is looked up in the new one exactly once, so the cost of a diff is the cost of n lookups in whichever collection you picked: O(n) for a HashSet or HashMap, O(n log n) for a BTreeSet or BTreeMap. Applying one costs the same, since each removal is a lookup rather than a rebuild.

§A map’s membership diff is a different shape

A map is a collection of entries, but one with a rule a set has no equivalent of: no two entries share a key. That rule earns a smaller delta, so a map field’s is an EntryDelta rather than a BagDeltaadd carries whole entries because the receiver needs to be told the value, while remove carries bare keys, since a key names an entry on its own.

use delta_struct::Delta;
use std::collections::BTreeMap;

#[derive(Delta)]
struct Deployment {
    #[delta_struct(field_type = "unordered")]
    labels: BTreeMap<String, String>,
}

let deployment = |labels: &[(&str, &str)]| Deployment {
    labels: labels.iter().map(|(k, v)| (k.to_string(), v.to_string())).collect(),
};

let delta = Delta::delta(
    deployment(&[("tier", "web"), ("zone", "a")]),
    deployment(&[("tier", "edge")]),
)
.unwrap();
// `tier` survived, so only what it holds now travels — the old value stays
// where it already is. `zone` left, and its key alone says so.
assert_eq!(delta.labels.add, vec![("tier".to_string(), "edge".to_string())]);
assert_eq!(delta.labels.remove, vec!["zone".to_string()]);

A key that survived with a new value is an addition, not a removal followed by one: applying an addition overwrites whatever the key held, so the removal would say nothing the addition does not already say.

Which shape a field gets is the collection’s business, not the field type’s — the delta field is declared as <T as Unordered>::Delta, and the collection’s Unordered impl picks. That is what lets one field type cover both without the derive needing to tell a map from a set.

This — not unordered-delta — is what to reach for when the values are scalars with no Delta impl of their own, which is the usual shape of a map of labels, tags, or config. The trade against unordered-delta is that a changed value travels whole rather than as its own delta; in exchange this asks nothing of the value type but PartialEq, and needs only TryIndex where unordered-delta needs TryIndexMut.

apply_delta preserves membership but not position — additions land wherever the collection decides to put them. Use ordered where that matters.

§unordered-delta

Like unordered, but for a collection of key/value entries whose values are worth diffing rather than resending. An entry whose key is on both sides is not a removal plus an addition: the two values are handed to Delta::delta and only the difference is recorded. The delta is a MapDelta, holding an add, a remove, and a change.

use delta_struct::{Delta, ScalarDelta};
use std::collections::HashMap;

#[derive(Delta)]
struct Service {
    port: u16,
    healthy: bool,
}

#[derive(Delta)]
struct Cluster {
    #[delta_struct(field_type = "unordered-delta")]
    services: HashMap<String, Service>,
}

let cluster = |port| Cluster {
    services: vec![("web".to_string(), Service { port, healthy: true })]
        .into_iter()
        .collect(),
};

let delta = Delta::delta(cluster(80), cluster(8080)).unwrap();
// `web` stayed put, so all that travels is the one field that moved.
assert!(delta.services.add.is_empty());
assert!(delta.services.remove.is_empty());
assert_eq!(delta.services.change[0].key, "web");
assert_eq!(delta.services.change[0].delta.port, ScalarDelta::Changed(8080));
assert_eq!(delta.services.change[0].delta.healthy, ScalarDelta::Unchanged);

The key is the collection’s own — the K of a HashMap<K, V> — not something you nominate. The field has to be a map: a HashMap or a BTreeMap. Formally it needs Extend and TryIndexMut, its entry type needs MapEntry (implemented for (K, V), which is what every std map iterates as), and its value type needs Delta.

TryIndexMut rather than TryIndex is what excludes sets here, and correctly so: applying a delta means mutating a value where it sits, which a set cannot allow without letting you invalidate the hash or ordering it filed the element under.

Every key of the old collection is looked up in the new one exactly once, so as with unordered the cost is n lookups — O(n) for a HashMap, O(n log n) for a BTreeMap. Applying one preserves membership rather than position, also the same as unordered.

§ordered

The field is diffed positionally with Myers’ algorithm, and the delta is a minimal edit script: a SeqDelta holding Splices that each say “at this index, drop this many items and put these in their place”.

use delta_struct::{Delta, Splice};

#[derive(Delta)]
struct Playlist {
    #[delta_struct(field_type = "ordered")]
    tracks: Vec<String>,
}

let old = Playlist { tracks: vec!["intro".to_string(), "b".to_string(), "outro".to_string()] };
let new = Playlist { tracks: vec!["intro".to_string(), "x".to_string(), "outro".to_string()] };

let delta = Delta::delta(old, new).unwrap();
assert_eq!(
    delta.tracks.splices,
    vec![Splice { at: 1, remove: 1, insert: vec!["x".to_string()] }],
);

Splice positions index the old sequence and arrive sorted and non-overlapping, so applying one is a single forward pass. Reordering is a real change here where unordered would see none, and applying a delta reproduces the new sequence exactly, position included.

The collection needs IntoIterator and FromIterator, and its items need Hash + Eq, because that is what indexing the sequences for Myers requires. This is the one field type that takes a Vec, and so the only one that will diff a sequence at all — but f64 is neither Hash nor Eq, so a Vec<f64> still has nowhere to go but scalar.

§delta

The field is itself diffed recursively, which keeps a nested change from resending the whole subtree. Requires the field’s type to implement Delta; the delta struct holds Option<<T as Delta>::Output>.

use delta_struct::{Delta, ScalarDelta};

#[derive(Delta)]
struct Inner {
    a: i32,
    b: i32,
}

#[derive(Delta)]
struct Outer {
    #[delta_struct(field_type = "delta")]
    inner: Inner,
    name: String,
}

let old = Outer { inner: Inner { a: 1, b: 2 }, name: "x".to_string() };
let new = Outer { inner: Inner { a: 1, b: 3 }, name: "x".to_string() };

let delta = Delta::delta(old, new).unwrap();
let inner_delta = delta.inner.expect("`b` changed");
assert_eq!(inner_delta.a, ScalarDelta::Unchanged);
assert_eq!(inner_delta.b, ScalarDelta::Changed(3));

§Enums

An enum can change in two ways a struct cannot, and its delta says which. Two values in the same variant are diffed field by field exactly as a struct is. Two values in different variants have no difference to describe — the new one shares nothing with the old — so the whole value travels.

That fork is EnumDelta, and Output becomes EnumDelta<Self, {Self}Delta> rather than the bare companion type. The generated {Self}Delta carries one variant per diffable source variant; a field-less variant gets none, since two of those can never differ.

use delta_struct::{Delta, EnumDelta, ScalarDelta};

#[derive(Delta)]
#[delta_struct(delta_leader = "#[derive(Debug)]")]
enum Shape {
    Empty,
    Circle { r: u32 },
}

// Same variant: only the field that moved travels.
let delta = Delta::delta(Shape::Circle { r: 1 }, Shape::Circle { r: 2 }).unwrap();
match delta {
    EnumDelta::Delta(ShapeDelta::Circle { r }) => assert_eq!(r, ScalarDelta::Changed(2)),
    _ => panic!("same variant"),
}

// Different variant: a replacement, not a difference.
let delta = Delta::delta(Shape::Empty, Shape::Circle { r: 3 }).unwrap();
assert!(matches!(delta, EnumDelta::Became(Shape::Circle { r: 3 })));

Keeping Became on a crate type rather than as an arm of the generated enum is what lets you have a variant of your own called Became.

§Why apply_delta returns a Result

A delta built while a value was one variant can arrive at a value that is now another. That is divergence, and it is the one thing applying a delta can genuinely fail at — hence Mismatch, which names the type, the variant the delta expected, and the variant it found.

let delta = Delta::delta(Shape::Circle { r: 1 }, Shape::Circle { r: 2 }).unwrap();
let mut diverged = Shape::Empty;
assert_eq!(
    diverged.apply_delta(delta),
    Err(Mismatch { type_name: "Shape", expected: "Circle", found: "Empty" }),
);

Nested deltas propagate the innermost mismatch rather than wrapping it, so what you get names the enum that actually disagreed rather than the outermost struct you called apply_delta on.

Only enums can produce this. A struct’s apply_delta returns Ok unless one of its fields is an enum, which is why the unwraps in the struct examples above are safe rather than sloppy.

§Container attributes

#[delta_struct(...)] on the struct itself accepts:

  • default = "..." — the field type used for fields without their own field_type. Defaults to "scalar".
  • delta_leader = "..." — tokens to emit immediately above the generated struct. This is how you attach derives, doc comments, or any other attribute to a type you never get to write by hand.
use delta_struct::{Delta, ScalarDelta};
use std::collections::HashSet;

#[derive(Delta)]
#[delta_struct(
    default = "unordered",
    delta_leader = "/// The changes to a `Tags`.\n#[derive(Debug)]"
)]
struct Tags {
    labels: HashSet<String>,
    // Opt an individual field back out of the container default.
    #[delta_struct(field_type = "scalar")]
    revision: u32,
}

let old = Tags { labels: HashSet::new(), revision: 1 };
let new = Tags {
    labels: vec!["new".to_string()].into_iter().collect(),
    revision: 2,
};
let delta = Delta::delta(old, new).unwrap();
assert_eq!(format!("{:?}", delta.labels.add), r#"["new"]"#);
assert_eq!(delta.revision, ScalarDelta::Changed(2));

delta_leader also works on individual fields, where it decorates the generated field instead of the generated struct.

#[derive(Delta)]
struct Host {
    #[delta_struct(delta_leader = "/// The new port, if it moved.")]
    port: u16,
}

§Working with serde

For scalar and delta fields there is no serde integration to enable; delta_leader is the whole story. Put the derives on the generated struct and it serializes like anything else:

use delta_struct::Delta;

#[derive(Delta)]
#[delta_struct(delta_leader = "#[derive(serde::Serialize, serde::Deserialize)]")]
struct Config {
    host: String,
    port: u16,
}

let old = Config { host: "localhost".to_string(), port: 80 };
let new = Config { host: "localhost".to_string(), port: 8080 };

// Sender: there is no message to send at all when nothing changed.
let payload = Delta::delta(old, new).map(|delta| serde_json::to_string(&delta).unwrap());
assert_eq!(payload.as_deref(), Some(r#"{"host":"unchanged","port":{"changed":8080}}"#));

// Receiver applies it to whatever it already had.
let mut config = Config { host: "localhost".to_string(), port: 80 };
config.apply_delta(serde_json::from_str::<ConfigDelta>(&payload.unwrap()).unwrap()).unwrap();
assert_eq!(config.port, 8080);

Field-level delta_leader carries serde attributes just as well, so skip_serializing_if can keep unchanged fields out of the payload entirely rather than sending them as null:

use delta_struct::Delta;

#[derive(Delta)]
#[delta_struct(delta_leader = "#[derive(serde::Serialize)]")]
struct Config {
    #[delta_struct(delta_leader = "#[serde(default, skip_serializing_if = \"::delta_struct::ScalarDelta::is_unchanged\")]")]
    host: String,
    #[delta_struct(delta_leader = "#[serde(default, skip_serializing_if = \"::delta_struct::ScalarDelta::is_unchanged\")]")]
    port: u16,
}

let old = Config { host: "localhost".to_string(), port: 80 };
let new = Config { host: "localhost".to_string(), port: 8080 };

let delta = Delta::delta(old, new).unwrap();
assert_eq!(serde_json::to_string(&delta).unwrap(), r#"{"port":{"changed":8080}}"#);

§Checking that a delta belongs

Delta::apply_delta assumes the value it is handed equals the old the delta came from, and checks nothing. Over an unreliable transport that assumption breaks: a message is dropped, delivered twice, or arrives at a receiver whose state drifted for some other reason, and the two sides diverge in silence.

Versioned is the opt-in fix. It pairs a value with a version counter and a Fingerprint of its contents, and refuses any delta that does not belong.

use delta_struct::{Applied, Delta, Fingerprint, Rejected, Versioned};

#[derive(Clone, Debug, Delta, Fingerprint, PartialEq)]
#[delta_struct(delta_leader = "#[derive(Clone)]")]
struct Config {
    host: String,
    port: u16,
}

let config = |port| Config { host: "localhost".to_string(), port };

let mut sender = Versioned::new(config(80));
let mut receiver = Versioned::new(config(80));

let first = sender.commit(config(8080)).expect("the port changed");
let second = sender.commit(config(9090)).expect("the port changed again");

// Delivered twice: recognised and ignored.
assert_eq!(receiver.apply(first.clone()), Ok(Applied::Updated));
assert_eq!(receiver.apply(first), Ok(Applied::Stale));
assert_eq!(receiver.apply(second), Ok(Applied::Updated));
assert_eq!(receiver.get(), sender.get());

// A delta from a stream this receiver never joined is refused rather than
// half-applied.
let mut stranger = Versioned::new(config(80));
let orphan = Versioned::new(config(1)).commit(config(2)).unwrap();
assert!(matches!(stranger.apply(orphan), Err(Rejected::Base { .. })));

Every VersionedDelta carries four numbers, each catching a failure the others cannot:

FieldCatches
from, toA message dropped, reordered, or replayed.
baseA receiver whose state drifted for any reason, including one that never came through this stream.
resultThe delta itself being wrong — mismatched schema versions, or a bug.

A rejected delta leaves the receiver untouched and its version unmoved, so a later delta in the same stream fails too rather than papering over the hole. The answer to any Rejected is to resend the whole Versioned, which serializes as a unit and carries the version the receiver resumes from.

None of this touches the Delta trait, the derive, or any generated struct. If you are diffing locally rather than over a wire, you never name anything in this section and pay for none of it.

§Fingerprint

Fingerprint is a separate derive because std::hash::Hash cannot do the job: it is not implemented for HashSet or HashMap — exactly the collections the unordered field types require — and its standard hasher is allowed to change between Rust releases, which would make a toolchain upgrade on one side of a connection look like corruption.

So sets and maps fold commutatively, iteration order cannot reach the result, and the hash is pinned to FNV-1a constants written down in the source. The same value fingerprints identically on any platform and any Rust version. Unlike Delta, it derives on enums too.

Checking costs a full traversal of the state on each commit and each apply — cheaper than serializing it, but not free, which is the price of the base and result guarantees.

§What gets generated

For struct Foo, deriving Delta emits struct FooDelta with the same visibility as Foo and the same generic parameters, carrying over their bounds and where clause as written. All of its fields are pub, and by default it derives nothing at all — reach for delta_leader whenever you need Debug, Clone, or serde on it. (Likewise if your crate sets #![deny(missing_docs)]: the generated struct and its fields need doc comments supplied through delta_leader.)

Every field type maps one source field onto exactly one delta field, so a delta struct always has the same fields in the same order as the struct it came from — only their types differ. A tuple struct’s delta is a tuple struct in turn, so its fields keep their positions:

use delta_struct::{Delta, ScalarDelta};

#[derive(Delta)]
struct Meters(i32);

let delta = Delta::delta(Meters(3), Meters(4)).unwrap();
assert_eq!(delta.0, ScalarDelta::Changed(4));

§Limitations

  • Unions are rejected. Structs and enums are both supported; a union has no way to say which of its fields is live, so there is nothing to diff.
  • An enum with no variants is rejected. An uninhabited type has no two values that could differ.
  • Every type parameter gets a PartialEq bound on the generated impl, whether or not the field that uses it needs one.
  • A unit struct’s delta is always None, as is that of a struct with no fields — there is nothing that could differ.
  • ordered items need Hash + Eq, so float sequences are out. See that section above.
  • A Vec cannot be an unordered field. Membership diffing goes through TryIndex, and a Vec has no sub-linear lookup to offer — implementing it would only hide a quadratic scan behind an O(1)-looking call. Use a HashSet or a BTreeSet, or ordered if position matters.
  • unordered-delta keys are the collection’s own. There is no way to nominate a field of the value as the key, so a Vec<Record> has to become a HashMap<Id, Record> to use it.
  • Versioned assumes one writer per stream. Two senders committing against the same base both produce from: 0, and the second is rejected rather than merged. Divergence is detected, not reconciled — reach for a CRDT if you need concurrent writers.

Re-exports§

pub use bag::BagDelta;
pub use entry::EntryDelta;
pub use fingerprint::fingerprint_of;
pub use fingerprint::Fingerprint;
pub use index::TryIndex;
pub use index::TryIndexMut;
pub use map::KeyedDelta;
pub use map::MapDelta;
pub use map::MapEntry;
pub use seq::SeqDelta;
pub use seq::Splice;
pub use unordered::Unordered;
pub use variant::EnumDelta;
pub use variant::Mismatch;
pub use version::Applied;
pub use version::Rejected;
pub use version::Versioned;
pub use version::VersionedDelta;

Modules§

bag
Membership diffing, behind the unordered field type when the field is a set.
entry
Keyed membership diffing, behind the unordered field type when the field is a map.
fingerprint
Stable content hashing, so a receiver can check that a delta is being applied to the state it was computed against.
index
Fallible lookup, the operation the unordered field types diff through.
map
Keyed diffing, behind the unordered-delta field type.
seq
Positional diffing, behind the ordered field type.
unordered
The bridge from a collection to the shape its membership diff takes.
variant
The delta of a value that might have changed shape, and the failure that comes with it.
version
Detecting a delta that is being applied to the wrong state.

Enums§

ScalarDelta
This type exists as a workaround for the serde data model being unable to distinguish between Some(None) and None in serialized bytes. It is, in nearly every respect, Option<T>, just with some more semantic clarity. You may freely convert between this and Option<T> using From::from or Into::into.

Traits§

Delta
Computing the difference between two values, and applying it to a third.

Derive Macros§

Delta
Derives Delta, generating a {Self}Delta struct that holds only the changed parts of a value plus the trait implementation that produces and applies one.
Fingerprint
Derives Fingerprint, a stable content hash used to check that a delta is being applied to the state it was computed against.