Skip to main content

Module cdc_runtime

Module cdc_runtime 

Source
Expand description

M11 FU-4 — change-data-capture (CDC) runtime.

Drives every registered uni_plugin::traits::cdc::CdcOutputProvider by subscribing to the commit broadcaster and converting each crate::notifications::CommitNotification into a uni_plugin::traits::cdc::CdcBatch delivered to every active stream.

§Lifecycle

  • At Uni::build time, CdcRuntime::spawn takes a snapshot of the registered CDC providers, loads each provider’s last committed LSN from the JSON-sidecar <data_path>/_system/cdc_checkpoints.json, and calls provider.start(CdcStartContext { from_lsn }) to obtain a live uni_plugin::traits::cdc::CdcStream. The runtime spawns a tokio task that subscribes to the commit broadcaster and forwards each commit as a CdcBatch to every stream.
  • Per-commit, after every stream has accepted the batch, the runtime calls checkpoint() on each stream and persists the returned LSN to the sidecar. On restart, providers resume from that LSN.
  • On shutdown the runtime calls shutdown() on each stream and exits.

§v1 limitations

CdcBatch::mutations ships as an empty single-row RecordBatch today — the LSN advancement, ordering, and checkpoint round-trip are the parts under test. Filling the batch with the actual mutation rows uses the same machinery as crate::triggers::MutationEvents and is tracked as a follow-up.

Structs§

CdcCheckpointSidecar
JSON-sidecar checkpoint store at <data_path>/_system/cdc_checkpoints.json.
CdcRuntime
Host-side CDC runtime that drives every registered provider on the commit broadcaster.
PersistedCheckpoint
Per-provider checkpoint row written to the JSON sidecar.