sqlite-diff-rs

A Rust library for building SQLite changeset and patchset binary formats programmatically.

Overview

SQLite's session extension defines a binary format for tracking and applying database changes. This crate constructs that binary data without linking SQLite, which is useful for offline sync (build a changeset on a server and apply it on a SQLite client), CDC pipelines (produce the input expected by sqlite3_changeset_apply() from your own change events), cross-database sync (convert PostgreSQL change streams from wal2json, Debezium, or Maxwell into the SQLite format), and generating test fixtures for changeset processing code.

This crate is not the SQLite sqldiff tool, which compares two existing database files. This library constructs the changeset and patchset binary format programmatically from your own change data.

Installation

Add to your Cargo.toml:

[dependencies]
sqlite-diff-rs = "0.1"

Quick Start

use sqlite_diff_rs::{DiffOps, Insert, PatchSet, SimpleTable};

// Define a table schema: "users" with columns (id, name), PK at index 0
let users = SimpleTable::new("users", &["id", "name"], &[0]);

// Build a patchset with an INSERT
let patchset = PatchSet::<_, String, Vec<u8>>::new()
    .insert(
        Insert::from(users)
            .set(0, 1i64).unwrap()      // id = 1
            .set(1, "Alice").unwrap()   // name = "Alice"
    );

// Encode to binary format
let bytes: Vec<u8> = patchset.into();

// Apply with sqlite3_changeset_apply() or sqlite3session_patchset_apply()

Features

Feature	Description
`testing`	Enables `rusqlite` integration for differential testing
`wal2json`	Parse PostgreSQL wal2json output into changesets
`pg-walstream`	Integration with `pg_walstream` crate
`debezium`	Parse Debezium CDC JSON events
`maxwell`	Parse Maxwell CDC JSON events

Enable features in Cargo.toml:

[dependencies]
sqlite-diff-rs = { version = "0.1", features = ["wal2json"] }

Binary Format Reference

Both formats share the same container structure: one or more table sections, each with a table header followed by change records. The key difference is how much old-row data each operation carries.

Changesets ('T' / 0x54) store the complete old state of every column. This makes them reversible, so INSERTs can be turned into DELETEs and vice versa.

Patchsets ('P' / 0x50) omit old values for non-PK columns, producing a smaller, forward-only encoding that cannot be reversed.

Aspect	Changeset (`'T'`)	Patchset (`'P'`)
INSERT	All column values	All column values
DELETE	All old column values	PK values only
UPDATE old	All old column values	PK values + Undefined for non-PK
UPDATE new	All new column values	All new column values
Reversible	Yes	No
Wire size	Larger (carries full old state)	Smaller (omits non-PK old values)

See the SQLite session extension docs for the full specification.

`no_std` Support

This crate is no_std compatible (requires alloc). It can be used in embedded or WebAssembly environments.

Live demo

A serverless, two-peer chat demo lives at https://lucacappelletti94.github.io/sqlite-diff-rs/. Each browser runs its own SQLite (via sqlite-wasm-rs) accessed through diesel-sqlite-session, and the two peers exchange changeset bytes directly over a WebRTC data channel. There is no backend, no signaling server, and no STUN configuration beyond Google's public STUN endpoint.

To connect two browsers:

Open the demo URL in tab A and pick a display name. Click Create room.
The tab shows an offer URL containing the full SDP in its fragment (#o=...). Send it to tab B (chat app, paste, QR code).
Tab B opens the URL, picks its own name, and clicks Generate reply code. Send the resulting base64 blob back to tab A by any means (paste, chat app, QR code).
Tab A pastes the reply code into the "Paste their reply" box and clicks Connect. The data channel opens.
Every message, edit, and delete in either tab is captured as a SQLite session changeset and applied on the other side. The diff inspector at the bottom shows the wire bytes parsed back through sqlite-diff-rs.

The demo source is at examples/web-demo/.

License

MIT License, see LICENSE for details.

sqlite-diff-rs 0.1.2