Skip to main content

Crate cdcflow

Crate cdcflow 

Source
Expand description

§cdcflow

A Change Data Capture (CDC) pipeline that captures row-level changes from PostgreSQL and MySQL, streaming them to Kafka, PostgreSQL, Apache Iceberg, or stdout.

The pipeline reads events from a source, batches them by transaction boundaries, writes each batch to a sink, and checkpoints progress in an offset store. On restart, the pipeline resumes from the last checkpoint.

§Modes

  • CDC — append-only changelog of row changes
  • Replication — live replica of source tables (Iceberg and Postgres sinks)

Modules§

admin
config
error
event
metrics
offset
pipeline
schema
sink
source
worker_http