Crate magic_sync 

Keeps partial copies of PostgreSQL tables on SQLite replicas, and brings back what the replicas write — offline-first, by comparing what each replica holds with what it should hold.

A table is a struct that derives Row and says how it travels, and a Codec says how its rows turn into bytes. On the server, a Server (feature postgres) notes every change in the writing transaction, plans per replica what it should hold against what it is known to hold, and sends the difference through your transport. On a replica, a Replica (feature sqlite) applies what arrives in one transaction, and sends up what the replica wrote.

The book builds a small system step by step and explains the engine from the ground up; the replica protocol is what a replica in any language must do.

A build of this crate is one side: enable postgres for the server or sqlite for a replica, each in a workspace of its own (Set up a project). These docs show the side they were built for; the book publishes both, the server’s and a replica’s.

Public dependencies

What a host passes to the engine or reads back is another crate’s type, and a new major version of any of them is a new major version of this crate:

• sqlx 0.8: the pools, connections and transactions a host hands the engine, and the database error (Error::Database, sqlx::Error).
• prost 0.14: a table that travels as protobuf is a message of it.
• uuid 1: replicas, partitions and keys are uuid::Uuids.
• chrono 0.4: the times a status carries are its DateTime<Utc>.
• serde 1: a table that travels as JSON (wire::Codec::json) is a row that is Serialize and Deserialize, and the dashboard’s documents are serialized with it.

Re-exports

pub use row::Columns;

pub use row::Row;

Modules

envelope

What the engine sends a replica and a replica sends back. A host carries them as bytes on whatever transport it has, and never looks inside: the messages are protobuf, as proto/sync.proto declares them for a replica in another language, and the replica protocol says what they mean. What a host sets is how large one may be: Limits.

plan

The planner: which rows each replica holds, and what to send it so it holds exactly those.

row

A table declared once, as the struct that is its row.

run

The engine, run for the host: the loops every host would otherwise write the same way, and get wrong the same ways. A host builds its side’s runner once — the Server on the server, a Replica on a replica — and runs it in a task of its own.

status

What the engine answers about itself, for a dashboard or an alarm: whether each replica holds what it should, how far along it is, where one row stands, and what waits for a fix. Every read takes a Scope, so a host shows each audience its own replicas and nothing else; the engine knows replicas only by id, and the host puts names on them.

wire

A mirrored table on the wire: its rows encoded as the table’s own message, and applied on the other side by the rules its row declares.

Enums

Error

What the engine can fail with. Error::is_transient says whether the same work may succeed as it is once the infrastructure is back.

Constants

SERVER_SCHEMA_VERSION

The version of the engine’s tables on the server’s side: the server refuses to run against a database that installed another. It counts the server’s own steps, independent of the replicas’ version.

Type Aliases

Conn

A connection of this side’s database: &mut *tx inside a transaction.

Derive Macros

Columns

A part of a row, flattened into it with #[sqlx(flatten)]: its columns are the row’s. Its fields take the same flags as a row’s; on the struct, only #[row(crate = "…")].

Row

One struct is one table, column for column, and says how it travels: implements Row. The same struct serves both sides.