Expand description
crabka-replicator — cross-cluster geo-replication for Crabka.
Re-exports§
pub use error::ReplicatorError;pub use error::Result;
Modules§
- admin_
util - Shared admin/IO helpers used by replicator engine tasks.
- checkpoint_
store - Position recovery: persist the worker’s
SourceOffsetto a compacted internal topic on the TARGET cluster, keyed by flow name. - config
- Replicator configuration model (clusters / flows / policies).
- error
- Replicator error type.
- mm2
- MirrorMaker-2 byte-compatible record codecs.
- naming
- Remote-topic naming and loop-prevention primitives.
- offset_
sync_ store - In-memory source->target offset translation built from MM2 offset-sync records. Translation is “at-or-before”: find the latest sync with upstream <= committed, then downstream + (committed - upstream). Never maps past un-replicated data.
- record
- The payload that travels through the connect runtime for one replicated record.
- residency
- Deny-wins residency gate. A topic-flow is permitted unless a matching policy
denies it. Semantics mirror Crabka’s ACL authorizer: DENY beats ALLOW; an
allow_zoneslist means “only these zones are allowed” (deny-by-default for the matched topic). - selector
- Include/exclude name selectors.
*is a glob wildcard; patterns are anchored. - sink
- Sink side: residency gate + naming + provenance loop-guard, produce to target, and emit source->target offset syncs.
- source
- Source side: a consumer on the source cluster that emits
ReplicatedRecords and snapshots all partition positions as aSourceOffset. - supervisor
- Supervisor: top-level task tree that owns and restarts replication workers.
- tasks
- Background tasks that run alongside the main replication workers.
- worker
- Per-flow replication worker: drives one directional connect pipeline
(
SourceConsumer->TargetSink) plus the heartbeat and checkpoint background tasks, with build-retry resilience and clean shutdown.