Skip to main content

Crate mysql_cdc_rs

Crate mysql_cdc_rs 

Source
Expand description

§mysql-cdc-rs

English | 简体中文

One dependency for MySQL CDC and replication. This facade re-exports mysql-canal and mysql-replication, preserving their types, lifetimes and behavior.

The processing layers are pikachu → mysql-canal → mysql-replication; this facade provides access to the two libraries without adding a processing stage. Use replication to subscribe to binlogs, parse protocol events and optionally inspect sparse raw row images with events::RowsDecoder<M>. Use canal::Canal for whitelist-based conversion, committed transaction handlers and checkpoint recovery. See the architecture guide for the event flow and boundaries.

To integrate from local source, use the dependency below; replace /path/to/mysql-cdc-rs with your checkout path.

[dependencies]
mysql-cdc-rs = { version = "2.2.1", path = "/path/to/mysql-cdc-rs/crates/mysql-cdc" }

The Rust import path is mysql_cdc_rs. The crate root only re-exports canal and replication. Use canal::Canal for the consumer lifecycle, canal::config::AppConfig for its configuration and canal::Error / canal::Result for its errors. Raw readers, streams, configuration and errors belong to replication. The aggregate has no consumer instance, configuration or error type of its own.

canal.toml uses the standalone canal configuration, beginning at [[sources]], with [sources.mysql-replication] under each source.

use mysql_cdc_rs::canal::{Canal, CommittedTransaction, Result, config::AppConfig};
use std::path::Path;

fn main() -> Result<()> {
    let mut canal = Canal::new(AppConfig::read(Path::new("canal.toml"))?)?;
    canal.add_handle(|transaction: CommittedTransaction<'_>| {
        for change in transaction.changes {
            println!(
                "{} {:?} {:?}",
                transaction.source_id, change.operation, change.pk
            );
        }
        Ok(())
    });
    canal.run()
}

Protocol events can be passed directly to CDC conversion without wrapping or copying them:

Raw events use replication::BinlogEvent<Full> / BinlogEvent<Minimal> to select image types statically, sharing marker types with canal. Generic readers bind TABLE_MAP automatically; image fields() provides column definitions and borrowed values. MINIMAL distinguishes absent fields from SQL NULL, while FULL requires every column in each existing image. Missing optional metadata such as column names stays unknown; canal changes expose logged before/after values with historical SQL ColumnMeta in fields.

use mysql_cdc_rs::{
    canal::{Change, Result, engine::converter::EventConverter},
    replication::{BinlogEvent, BinlogMode},
};

fn convert<M: BinlogMode>(
    converter: &mut EventConverter,
    event: &BinlogEvent<'_, M>,
) -> Result<()> {
    converter.convert_binlog(event, &mut |change: Change<'_>| {
        println!("{:?} {:?}", change.operation, change.pk);
        Ok(())
    })?;
    Ok(())
}

EventConverter emits row changes while processing ROWS, before transaction commit, and returns an EventAction describing control boundaries. The example above only demonstrates conversion. Applications delivering committed transactions should use Canal or feed every event through Processor; direct conversion callbacks do not buffer transactions, discard rolled-back changes or persist checkpoints. Feed the file’s FDE before converting its row events.

For offline raw events, import mysql_cdc_rs::replication::file::BinlogFile. For network replication, import mysql_cdc_rs::replication::stream::ReplicationStream. See listen.rs and dump_file.rs for complete examples. From the workspace root:

cargo run -p mysql-cdc-rs --example canal_listen -- canal.toml
cargo run -p mysql-cdc-rs --example replication_dump_file -- /path/to/binlog.000001

CDC uses named sources with independent schemas, whitelists and checkpoints. It delivers only committed changes containing the source, database/table, operation, exact primary key, changed column names and before/after fields with metadata and values. Rollback discards buffered changes; failures stop consumption, and a crash before checkpoint persistence may cause replay. There is no default output handler in the library. The pikachu package provides the pikachu-cdc CLI.

The lower-level crates remain available as separate dependencies. See the project guide, CDC library and replication library for configuration, output contracts, version support and limitations.

Event reference · Development · Testing

Re-exports§

pub use mysql_canal as canal;
pub use mysql_replication as replication;