Crate sources_core 

The source abstractions for flusso.

A source has two independent responsibilities, each its own module. Neither module references the other; the engine is the only thing that bridges them.

• cdc — what changed? A pluggable change-capture mechanism that yields a stream of thin Changes and confirms progress via an Ack. Logical replication (WAL) is the first mechanism; polling or triggers can follow.
• document — what to build? Turns a changed row (named by table and key) into the target documents it affects, and assembles each one.

Alongside them, validate_indexes is the source-independent half of a check every source can offer: a self-describing schema states its own types, so the mapping is derived without a database, and a reachable database is used only to confirm the declared types and nullability match the real columns. A source supplies only the one store-specific piece — a Catalog over its column types.

CaptureProvisioning is a second such offer: given the tables an index reads (SourceSpec::all_tables), a source reports whether it can stream them and — when privileged — provisions the gap (for Postgres, a publication). Mechanism-neutral, so the daemon/CLI drive it without naming “publication”.

Both build on two shared, mechanism-neutral primitives that belong to neither concern:

• RowKey — a row’s primary key as ordered column/value pairs.
• SnapshotTable — a schema-qualified table the engine asks a mechanism to snapshot when seeding an index.
• SourceSpec — the source’s own view of what to build (the enabled indexes and their schemas), a subset of the top-level config translated by the composition root so the backend never sees Config.
• SourceError / Result — the common error type.

Keeping the two abstractions apart means a deployment can mix any change mechanism with any document builder, and either can be implemented, tested, or replaced without touching the other.

Modules

cdc

Change data capture: what changed?

document

Document construction: what to build?

Structs

ColumnInfo

How a store reports one base-table column: its native type name (as the store spells it, e.g. Postgres character varying(255)) and whether it admits null.

CoverageReport

What a source found when asked whether it can stream a set of tables.

Diagnostic

One disagreement between a declared schema and the live database.

QualifiedTable

A schema-qualified table, the unit a source needs to reason about coverage (which tables it must be able to stream). Ordered by (schema, table) so a BTreeSet of them is deterministic.

RowKey

A row’s primary key, as ordered column/value pairs.

SnapshotTable

A schema-qualified table to snapshot during an initial backfill.

SourceSpec

The enabled indexes a source must build, each paired with its schema.

Enums

Severity

How serious a Diagnostic is.

SourceError

Why a ChangeCapture failed to start or to produce the next change.

Traits

CaptureProvisioning

A source’s ability to report and provision the prerequisites for streaming a set of tables. Implemented per mechanism (Postgres backs it with a publication); consumed by the CLI (check reports, run ensures) only through this neutral surface.

Catalog

A source’s view of its own catalog: the type and nullability of a base-table column. This is all validate_indexes needs from a source to check a declared schema against the live store.

Functions

validate_indexes

Validate every index in spec against the store behind catalog, returning the disagreements found. An empty result means the declared schema matches the database. The spec already holds only enabled indexes.

Type Aliases

Result

Result alias for source operations.