Expand description
kevy-replicate — primary-to-replica streaming replication.
One primary streams every applied mutation to N read replicas over a long-lived TCP connection, using a RESP3-extended frame format with an offset envelope. New replicas join via an inline snapshot ship, then catch up from the live frame stream.
wire— RESP-based frame format (seedocs/wire.md).wire_snapshot(internal) — snapshot-ship framing for the joining replica.source— primary-side bounded backlog indexed by offset.handshake—REPLICATE FROM <offset> ID <id>parse ++ACKformat.slot— per-replica state + reconnect-window expiry.replica— replica-side blocking TCP client (handshake + frame-decoding iterator).
§Applying replicated frames
ReplicaClient yields decoded (offset, Argv) tuples; applying
them to a local store is the caller’s responsibility — the right
dispatcher depends on where the replica’s data lives. The wire
format intentionally carries the exact RESP argv the primary
applied, so any dispatcher that hands Argv through Redis-verb
routing produces a byte-equivalent local store.
A replica against a primary built from this crate’s primary-side
pieces — the backlog (source), the handshake parser and +ACK
(handshake) — applying each frame to a map:
use std::collections::HashMap;
use std::io::{Read, Write};
use kevy_replicate::feed::FeedPosition;
use kevy_replicate::handshake::{HandshakeReq, encode_ack};
use kevy_replicate::replica::ReplicaClient;
use kevy_replicate::source::ReplicationSource;
// primary: two applied writes sit in the backlog
let mut backlog = ReplicationSource::new(1 << 20);
for cmd in [["SET", "a", "1"], ["SET", "b", "2"]] {
backlog.push_mutation(&kevy_resp::Argv::from(cmd.map(|w| w.as_bytes().to_vec()).to_vec()));
}
let listener = std::net::TcpListener::bind("127.0.0.1:0")?;
let addr = listener.local_addr()?;
let primary = std::thread::spawn(move || -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
let (mut sock, _) = listener.accept()?;
let (mut buf, mut argv) = (Vec::new(), kevy_resp::Argv::default());
while kevy_resp::parse_command_into(&buf, &mut argv)?.is_none() {
let mut chunk = [0u8; 256];
let n = sock.read(&mut chunk)?;
buf.extend_from_slice(&chunk[..n]);
}
let req = HandshakeReq::parse(&argv)?;
sock.write_all(&encode_ack(FeedPosition::new(1, req.from.offset)))?;
for frame in backlog.frames_from(req.from.offset).map_err(|e| format!("{e:?}"))? {
sock.write_all(&frame.bytes)?;
}
Ok(()) // closing the socket ends the stream
});
// replica: apply every frame in offset order
let mut store = HashMap::new();
for frame in ReplicaClient::connect(addr, "replica-a", 0)? {
let frame = frame?;
if let [b"SET", key, value] = frame.argv.iter().collect::<Vec<_>>()[..] {
store.insert(key.to_vec(), value.to_vec());
}
}
primary.join().expect("primary thread").map_err(|e| e.to_string())?;
assert_eq!(store.get(&b"b"[..]).map(Vec::as_slice), Some(&b"2"[..]));
assert_eq!(store.len(), 2);In kevy itself the dispatcher is kevy::KevyCommands::dispatch
over a kevy::KeyspaceStore: the frames carry the exact argv the
primary applied, so the replica’s store ends up byte-equivalent.
§Features
secure(default) — Noise IK encrypted, mutually authenticated links:replica::ReplicaSecurityandreplica::ConnectOptions::with_security. Off, a replica connects in plaintext only and the handshake code is not linked.
See the replica_apply_dispatch_mirrors_primary_store integration
test in crates/kevy/tests/replication.rs for the pattern under
the full primary+replica end-to-end harness.
The kevy binary also ships full server-as-replica mode (it
auto-spawns a ReplicaClient when [replication] role = "replica",
routing frames into the reactor with re-replication suppression);
the in-process recipe above is for any user that wants to drive
replication themselves.
Modules§
- feed
- CDC feed layer over
crate::source::ReplicationSource— the(generation, offset)cursor semantics the public FEED.* /changes_sincesurfaces speak. - handshake
- Replication handshake — the first round-trip a replica makes against the primary’s replication TCP listener.
- replica
- Replica-side client — connect to a primary’s replication listener, perform the handshake, then yield decoded mutation frames in order.
- slot
- Per-replica slot bookkeeping for the primary’s streaming loop.
- source
- Primary-side replication source — bounded backlog of recent mutations, indexed by monotonic offset.
- wire
- Wire format for replicated mutations — see
docs/wire.mdfor the full spec.