Crate faucet_source_spanner

Expand description

Google Cloud Spanner query source connector for faucet-stream.

Runs a GoogleSQL query against a Spanner database via the streaming read API (ExecuteStreamingSql) and emits rows as JSON with bounded memory. Supports named bind parameters, stale reads, incremental replication via a monotonic column bookmark, live dataset discovery (INFORMATION_SCHEMA), and PK-range sharding for clustered execution.

source:
  kind: spanner
  config:
    project_id: my-project
    instance: my-instance
    database: my-db
    query: "SELECT * FROM orders WHERE updated_at > TIMESTAMP(@bookmark)"
    replication:
      type: incremental
      column: updated_at
      initial_value: "1970-01-01T00:00:00Z"

Authentication defaults to Application Default Credentials; a service account key can be supplied by path or inline via auth. Set emulator_host to target the Spanner emulator.

Structs§

ShardConfig: Primary-key range sharding settings for the Spanner source (clustered Mode B execution).
SpannerConnection: Connection block shared by the Spanner source and sink configs (flattened into each connector config via #[serde(flatten)]).
SpannerSource: Google Cloud Spanner query source.
SpannerSourceConfig: Configuration for SpannerSource.

Enums§

SpannerCredentials: Credentials used to authenticate against Cloud Spanner.
SpannerReplication: How the source replicates rows across runs.