Skip to main content

Crate walship

Crate walship 

Source
Expand description

Ship libSQL WAL frames to a warm replica on another machine.

walship keeps a copy of each SQLite database on a second machine, a batch of frames behind the original. It reads committed frames out of the primary’s WAL and writes them into the replica’s WAL with the frame APIs libSQL adds to SQLite. No SQL is replayed and there’s no log format of its own: the replica’s WAL ends up holding the same pages as the primary’s.

There are two halves:

  • A Shipper runs next to your databases. You tell it which ones exist and when they’re written, and its threads post frames to the peer.
  • A Receiver keeps the replicas in a directory. server::Routes serves one over HTTP with axum. To turn a replica back into a database you can open and write, call Receiver::take.

Each node ships to one peer, and a receiver accepts from any number of shippers, so nodes can pair up, form a ring, or all ship to one standby.

§Requirements

Your databases must be opened through libSQL’s build of SQLite (the libsql-ffi crate this crate links), in WAL mode, with automatic checkpoints turned off on every connection. SQLite would otherwise fold frames into the main file before they’re shipped. disable_checkpoint does that for a raw sqlite3 handle. The libsql crate doesn’t expose one; examples/with_libsql.rs shows what to do instead. Linking a second copy of SQLite into the same process breaks SQLite’s file locks; don’t.

§Example

let shipper = walship::Shipper::builder("/var/lib/myapp/db", "http://standby.internal:7350")
    .token(std::env::var("WALSHIP_TOKEN").expect("WALSHIP_TOKEN is set"))
    .start(rt)?;
shipper.track("orders");
// ...after each write to orders.db:
shipper.wake("orders");

§Features

The Receiver and the wal primitives are always built.

Re-exports§

pub use receiver::Receiver;
pub use shipper::Owner;shipper
pub use shipper::Shipper;shipper
pub use wal::disable_checkpoint;
pub use wal::RawDb;

Modules§

proto
JSON bodies of the HTTP protocol.
receiver
The receiving side: warm copies of databases that live somewhere else.
serverserver
axum routes that put a Receiver on the network.
shippershipper
The sending side: keeps a peer’s replica of each database here within a batch of the local WAL.
wal
The WAL primitives everything else is built on.

Enums§

Error
Everything that can go wrong in walship.

Functions§

valid_name
Database names become file names, so they’re kept to a safe alphabet: ASCII letters, digits, _, - and ., not starting with a dot, at most 128 bytes.

Type Aliases§

Result