Crate datum_cdc 

datum-cdc

datum-cdc exposes PostgreSQL logical replication as Datum streams.

The MVP supports PostgreSQL pgoutput protocol v1 with streaming transactions off, two-phase commit off, and text-mode tuple values. A CdcSource builder materializes a single-owner Tokio replication carrier that feeds a bounded Datum Source<ChangeEvent, CdcHandle>.

use datum_cdc::{CdcSource, PostgresCdcConfig, SlotLifecycle};

let source = CdcSource::postgres()
    .connect(PostgresCdcConfig::from_url(
        "postgresql://datum_cdc@127.0.0.1:55433/datum_cdc",
    )?)
    .slot("datum_orders")
    .publication("orders_pub")
    .slot_lifecycle(SlotLifecycle::Existing)
    .buffer_capacity(8192)
    .build()?;

Delivery And Feedback

The source is at-least-once. PostgreSQL standby feedback advances only after downstream code calls CdcCheckpointHandle::checkpoint for every event in the next contiguous committed transaction. If a process crashes after delivery but before checkpoint feedback, PostgreSQL may replay those rows. Downstream can deduplicate with (slot, tx_end_lsn, xid, event_index).

build_with_context() returns a SourceWithContext<ChangeEvent, CdcOffset, _> so the offset stays attached through context-preserving operators.

WAL Retention

Backpressure is the bounded Datum channel. When downstream stalls, the carrier stops reading ahead and PostgreSQL retains WAL until feedback advances. Operators should monitor CdcHandle::lag() and server-side pg_replication_slots (restart_lsn, confirmed_flush_lsn, wal_status, safe_wal_size where available). Use max_slot_wal_keep_size on PostgreSQL to bound blast radius; if PostgreSQL marks a slot lost, create a new slot after a fresh snapshot.

Slot Lifecycle

SlotLifecycle::Existing is the production default. CreateIfMissing validates or creates a pgoutput slot but never drops it. CreateOwned and Temporary may be dropped through CdcHandle::drop_slot; all other modes require force_drop_slot.

Deferred Scope

Initial snapshots, pgoutput protocol v2-v4 streaming/parallel/two-phase messages, binary decoding, schema-DDL capture, multi-slot merge ordering, and a provider-compatible rustls replication build are intentionally deferred. The current workspace disables pgwire-replication’s TLS feature because it forces rustls/ring while Datum’s network stack uses rustls/aws-lc-rs under the workspace --all-features gate.

Structs

CdcCheckpointHandle

Handle used by downstream code after it has durably accepted an event.

CdcHandle

Materialized management handle for a CDC source.

CdcHealth

CdcLag

CdcOffset

Stable offset for downstream checkpointing and de-duplication.

CdcSource

Entry point for CDC source builders.

ChangeEvent

One row/table change from PostgreSQL logical replication.

ColumnMetadata

Column metadata as sent by a pgoutput Relation message.

FileCheckpointStore

File-backed checkpoint store.

MemoryCheckpointStore

In-memory checkpoint store useful for tests and embedded ephemeral readers.

PgLsn

PostgreSQL log sequence number.

PostgresCdcBuilder

Builder for a PostgreSQL pgoutput source.

PostgresCdcConfig

PostgreSQL connection settings for datum-cdc.

ReconnectSettings

Bounded reconnect policy used by the carrier task.

RelationMetadata

Relation metadata cached by relation OID.

RowData

Row tuple values in relation-column order.

SourceMetadata

Metadata identifying the PostgreSQL replication source that produced an event.

TransactionMeta

Transaction metadata attached after the commit boundary is known.

TruncateOptions

Options attached to a pgoutput truncate message.

Enums

CdcError

Error type for PostgreSQL CDC source construction, administration, parsing, checkpointing, and carrier failures.

CdcStart

Starting point for a materialized CDC source.

CdcTlsConfig

TLS mode for the replication connection.

ChangeOperation

CDC operation kind.

ColumnValue

A text-mode tuple value from pgoutput.

FeedbackMode

Feedback policy. The MVP intentionally exposes only checkpoint-gated feedback because immediate feedback would violate at-least-once delivery.

ReplicaIdentity

PostgreSQL replica identity for a relation.

SlotLifecycle

Logical replication slot lifecycle policy.

Constants

VERSION

The datum-cdc crate version.

Traits

CdcCheckpointStore

Durable checkpoint store used to resume a source after process restart.

Type Aliases

CdcResult

Result alias used by datum-cdc.