Skip to main content

Module sync

Module sync 

Source
Expand description

Phase 3 merge engine: changeset build/apply, per-table sync rules, cursor + ack bookkeeping, and the file blob channel.

Every table that syncs declares itself in the registry below — its cursor rule (how exports resume) and its apply rule (how imports land). The rules are deliberately small:

  • LWW (updated_at cursor): the incoming row wins iff (updated_at, pk…) > (local updated_at, pk…). RFC3339 timestamps compare lexically, so both devices compute the same winner with no origin columns. A tie (same row written in the same nanosecond on two clocks — the clock-skew residual) keeps the local row on both sides; it is stable, never flip-flops, and is accepted per the Phase 1 decision.
  • Append ((created_at, id) tuple or AUTOINCREMENT cursor): INSERT OR IGNORE by the row’s sync identity (uuid id, or sync_id for the AUTOINCREMENT tables). Idempotent union — the exactly-once backstop when a changeset is re-imported after a crash.
  • Tombstones (AUTOINCREMENT cursor): applied destructively — a session tombstone cascades its messages and sources, a space tombstone removes the space row and its directory, a file tombstone removes the row and the local blob.
  • swarm_personas has no cursor of its own: it is versioned by the owning session (Phase 1 decision). Persona rows travel attached to their session row and are applied only when that session row wins LWW, as a wholesale roster replace.

Cursor lifecycle (the ack design): an export does not advance push_cursor — the receiver imports, then replies with an ack carrying its new pull cursors, and only then does the sender advance push_cursor. Idempotent apply is the backstop for crashes mid- exchange.

Structs§

ApplySummary
What applying a changeset did — the receiver’s log line, plus the blobs the transport still has to fetch.
Changeset
One device’s export, possibly carrying an ack for the device it was sent to. Serde JSON over the wire (or inside a zip bundle with blobs).
FileChange
PeerCursor
RowChange
TableSpec
One syncable table’s cursor + apply rules.
Tombstone

Enums§

Cursor
How a table’s rows are ordered and resumed across changesets.

Constants§

TABLES
The engine’s registry, in apply order (sessions before their children, spaces/files before messages so foreign rows land after their parents).

Functions§

apply_changeset
Apply one changeset: rows (per the registry), tombstones, file blobs, and the embedded ack. Returns the summary plus the reply cursors — the receiver’s new pull cursors per table, to be acked back to the sender. blob_source, when given, is a directory whose legacy blobs/<space_id>/<name> or content-addressed blobs/by-hash/<hash> path holds the sender’s payloads; missing blobs are reported instead.
build_ack
The ack for peer_id: this device’s pull cursors for that peer’s data — “I imported your rows up to here”. The transports embed it in their next changeset so the peer can advance its push cursors; without it the peer would re-export everything forever.
build_changeset
Build this device’s export for peer_id — every row past the peer’s acked cursor per table (no cursor → full export, the first-run bootstrap), the un-acked tombstones, and the file manifest. Does not advance any cursor: that happens only when the peer acks.
device_name
The human-readable device name riding in every changeset.
export_blobs
Copy the local blobs of a changeset’s manifest into dest/blobs/ — the transport’s blob channel. Never re-sends identical content: the receiver hash-checks before pulling.
put_blob
Store one HTTP/transport blob after checking that its manifest row exists, the declared size matches, and the content hash is exact. Metadata always wins first; an upload can never create a new file row or escape its space.
read_blob
Read a manifest-backed blob for an HTTP download. A missing or stale local file is None, never an unverified byte stream.
unpack_bundle
Unpack a bundle written by write_bundle into dest_dir (as blobs/…), returning the changeset. Entry paths are validated — a bundle must never write outside dest_dir.
write_bundle
Write a changeset plus its blobs as a zip: changeset.json, legacy blobs/<space_id>/<name> entries, and content-addressed blobs/by-hash/<hash> entries. Returns how many blobs were included.