crabka-connect-postgres 0.3.7

Postgres logical-decoding source connector for Crabka Connect
docs.rs failed to build crabka-connect-postgres-0.3.7
Please check the build logs for more information.
See Builds for ideas on how to fix a failed build, or Metadata for how to configure docs.rs builds.
If you believe this is docs.rs' fault, open an issue.

crabka-connect-postgres

crabka-connect-postgres is a Postgres logical-decoding source connector for crabka-connect.

The connector reads pgoutput changes from a logical replication slot and emits Kafka Connect-style records with Protobuf-framed keys and values. DELETE events emit a Protobuf-framed key with a None value, which represents a tombstone for compacted topics.

Offsets and acknowledgement

The connector tracks Postgres LSNs with SourceOffset. The offset partition contains the database and replication slot, and the offset position contains the LSN.

The live source reads with pg_logical_slot_peek_binary_changes, so polling does not advance the replication slot. After the runtime writes the sink record, commits the sink, and durably saves the checkpoint, it calls Source::acknowledge. The connector then advances the Postgres slot to the acknowledged LSN.

Setup expectations

This slice expects:

  • A pgoutput logical replication slot.
  • A publication covering the configured tables.
  • Configured table names for the source.
  • No initial snapshot support. Only changes available through logical decoding are emitted.

When table names are configured, the connector creates the publication if it is missing and validates that the publication covers the configured tables. It also creates the logical slot if it is missing.