Module registry 

Feature-gated dispatch from a string type to a concrete connector.

Every arm in this file is guarded by the matching source-* / sink-* Cargo feature so users can build a slim binary with just the connectors they need. The string keys here are the public contract of the CLI’s type: field in YAML/JSON pipeline configs.

Structs

PluginRegistry

A registry of connector factories keyed by their YAML type: string.

Constants

CLEANUP_SINK_KINDS

Sink kinds that implement scoped cleanup (Sink::cleanup_scope, #478) — deleting destination rows inside a source’s declared completeness scope that the run did not write. Kept in sync with each sink’s supports_cleanup() override; cli/tests/registry_capability_parity.rs asserts they agree.

DISCOVER_SOURCE_KINDS

Source kinds that implement live dataset discovery (Source::discover, issue #211) — mirrors the discoverable-source list in the connector docs. Single source of truth for the conformance scorecard (#330).

EXACTLY_ONCE_SOURCE_KINDS

Source connector kinds that deterministically replay (exactly-once-capable). Mirrors Source::supports_exactly_once overrides — keep in sync when a new source opts in. The single source of truth for both the boolean gate and the human-readable list shown in error messages (F44). kafka qualifies because partitions are immutable logs and every page carries a complete offsets bookmark (#291).

IDEMPOTENT_SINK_KINDS

Sink connector kinds that can durably commit a token atomically with data. Mirrors Sink::supports_idempotent_writes overrides — keep in sync when a new sink opts in. Single source of truth for the gate + the error-message list (F44).

OVERWRITE_SINK_KINDS

Sink kinds that support write_mode: overwrite (#492) — full-destination replacement via the atomic begin/commit staging lifecycle. Mirrors each sink’s Sink::supported_write_modes() override (which must list WriteMode::Overwrite); cli/tests/registry_capability_parity.rs asserts they agree. A subset of UPSERT_SINK_KINDS — overwrite lands on the keyed-write sinks as their lifecycle is added.

SCHEMA_EVOLUTION_SINK_KINDS

Sink kinds that can apply additive/widening DDL via Sink::evolve_schema. Mirrors each sink’s supports_schema_evolution() override. Iceberg is additive-only (new columns) via iceberg-rust 0.10.0’s update_schema action (#255).

SCOPED_OVERWRITE_SINK_KINDS

Sinks that support a scoped/windowed overwrite (#518) — replacing only the rows matching a scope (a date window) instead of the whole table. A subset of OVERWRITE_SINK_KINDS; the others still support full overwrite. (v1: the atomic scoped delete + staged insert; the wider SQL family and key-scope are follow-ups.)

STAGED_LOAD_SINK_KINDS

Sink kinds that bulk-load via an object-store stage + native load command (staged bulk load, #528). These mirror each sink’s Sink::supports_staged_load override. Redshift (write_strategy: copy), Snowflake (bulk_load), and BigQuery (bulk_load) load from S3 / an external stage / GCS respectively; ClickHouse pulls a staged S3/GCS object with the s3() / gcs() table function, and MSSQL/Synapse pulls a staged Azure Blob object with COPY INTO (staging: blocks, #528). The load SQL/URL generators are unit-tested; the server-side execution is not exercised in CI (no live warehouse + object store), same as Redshift/Snowflake/BigQuery.

UPSERT_SINK_KINDS

Sink kinds that support write_mode: upsert|delete. Mirrors each sink’s Sink::supported_write_modes() override. Single source of truth for the gate

Functions

build_sink

Build a Sink trait object from a (kind, config) pair. When the config carries auth: { ref: <name> }, the named provider is resolved from auth (the catalog) and injected into the connector.

build_source

Build a Source trait object from a (kind, config) pair. When the config carries auth: { ref: <name> }, the named provider is resolved from auth (the catalog) and injected into the connector.

sink_descriptions

One-line summary of every sink connector — the compiled-in built-ins plus any third-party connectors registered via PluginRegistry. Used by faucet list.

sink_exists

Check if a sink kind is registered (not unknown or disabled by feature gate).

sink_guarantee

The strongest delivery guarantee a sink kind can uphold (Sink::sink_guarantee, issue #292). Derived from IDEMPOTENT_SINK_KINDS / UPSERT_SINK_KINDS.

sink_kinds

Names of every compiled-in sink connector.

sink_schema

Return the JSON Schema for the named sink’s config struct.

sink_supported_write_modes

Write modes each sink kind supports. Kept in sync with each sink’s Sink::supported_write_modes() override via UPSERT_SINK_KINDS.

sink_supports_cleanup

See CLEANUP_SINK_KINDS.

sink_supports_idempotent_writes

See IDEMPOTENT_SINK_KINDS.

sink_supports_overwrite

Whether a sink kind supports write_mode: overwrite.

sink_supports_schema_evolution

See SCHEMA_EVOLUTION_SINK_KINDS.

sink_supports_scoped_overwrite

Whether a sink kind supports a scoped overwrite scope.

sink_supports_staged_load

Whether a sink kind supports staged bulk load. See STAGED_LOAD_SINK_KINDS.

source_descriptions

One-line summary of every source connector — the compiled-in built-ins plus any third-party connectors registered via PluginRegistry. Used by faucet list.

source_exists

Check if a source kind is registered (not unknown or disabled by feature gate).

source_kinds

Names of every compiled-in source connector.

source_replay_guarantee

The typed replay capability a source kind advertises (Source::replay_guarantee, issue #292). Derived from EXACTLY_ONCE_SOURCE_KINDS — the kind table stays the single source of truth; this is the typed view the delivery-guarantee derivation consumes.

source_schema

Return the JSON Schema for the named source’s config struct.

source_supports_discover

Whether a source kind supports faucet discover (dataset introspection).

source_supports_exactly_once

See EXACTLY_ONCE_SOURCE_KINDS.

Type Aliases

SchemaFn

A closure returning the JSON Schema for a custom connector’s config. Powers faucet schema source <name> / faucet schema sink <name> for third-party connectors without instantiating one. Typically || schema_for!(MyConfig).

SinkFactory

A factory that builds a Sink trait object from its sink.config. See SourceFactory for the contract.

SourceFactory

A factory that builds a Source trait object from its JSON/YAML config (source.config). This is the extension point third-party connectors plug into: a custom-CLI author registers one per connector type: they want their faucet binary to understand.