Expand description
§flusso-sources-postgres
The Postgres logical-replication source for flusso: capture row changes over the WAL, backfill existing rows on first seed, and build each index document from the schema’s joins and aggregates.
§At a glance
Trait (sources-core) | Role |
|---|---|
ChangeCapture | streams thin per-row changes (table + primary key) from a replication slot; confirms progress via an LSN watermark |
DocumentBuilder | resolves which documents a changed row affects, then assembles each one. Also implements Catalog (a column’s SQL type + nullability, for index validation) |
CaptureProvisioning | reports stream coverage + a privilege verdict for an index’s tables, and provisions the gap (create/extend a publication) when allowed |
Re-exports (so callers build a capture without depending on pgwire-replication
directly): Lsn, ReplicationConfig, SslMode, TlsConfig.
💡 Did you know — flusso only reads the WAL; it never polls your tables, so follow-mode load on Postgres stays flat regardless of table size.
§The traits
ChangeCapturestreams thin per-row changes (table + primary key) from a replication slot and confirms progress via an LSN watermark. The slot advances only as far as the engine has durably written — at-least-once.DocumentBuilderresolves which documents a changed row affects and assembles each one. It also implementsCatalog, the store-specific half of index validation (a column’s SQL type + nullability).CaptureProvisioningtakes the tables an index reads, reports stream coverage and a privilege verdict, and — when allowed — provisions the gap by creating or extending a Postgres publication.
§Module layout
cdc/— change capture. Decodes the pgoutput message stream (the crate-private decoder, fuzzed via thefuzzingfeature), tracks relation metadata to extract each changed row’s key, buffers per transaction and emits on commit tagged with the commit LSN, manages the replication slot and publication, and snapshots tables for backfill.document/— document assembly. Resolves a changed row back to the document keys it affects (reverse resolution per join kind), generates one server-side SQL query per document (nested relations assembled withjson_build_object/json_agg, so there’s no N+1), and decodes the result into the value tree.
Structs§
- Lsn
- PostgreSQL Log Sequence Number.
- PgDocument
Builder - Builds index documents from a Postgres database, driven by a
SourceSpec— the enabled indexes and their schemas, translated from the top-level config by the composition root. Cheap to clone — the pool, spec, and primary-key cache are shared. - Replication
Config - Configuration for PostgreSQL logical replication connections.
- TlsConfig
- TLS/SSL configuration for PostgreSQL connections.
- WalChange
Capture - Postgres change capture over logical replication (pgoutput).
Enums§
- SslMode
- SSL/TLS connection mode.