Skip to main content

Crate faucet_sink_spanner

Crate faucet_sink_spanner 

Source
Expand description

Google Cloud Spanner sink connector for faucet-stream.

Writes JSON records to a Spanner table as batched mutations: Insert for append, InsertOrUpdate for write_mode: upsert, and Delete mutations for write_mode: delete / delete_marker rows. Exactly-once delivery (delivery: exactly_once) commits each page atomically with a _faucet_commit_token watermark row in one read-write transaction.

sink:
  kind: spanner
  config:
    project_id: my-project
    instance: my-instance
    database: my-db
    table_name: events
    write_mode: upsert       # append (default) | upsert | delete
    key: [id]                # must equal the table's PRIMARY KEY
    auth:
      type: application_default

Values are encoded against the destination column types read from INFORMATION_SCHEMA; commits are chunked at batch_size rows and a conservative 60,000-mutated-cell budget (Spanner’s hard limit is ~80,000 cells per commit including secondary-index amplification).

Structs§

SpannerConnection
Connection block shared by the Spanner source and sink configs (flattened into each connector config via #[serde(flatten)]).
SpannerSink
A sink that writes JSON records to a Cloud Spanner table.
SpannerSinkConfig
Configuration for the Cloud Spanner sink.

Enums§

SpannerCredentials
Credentials used to authenticate against Cloud Spanner.

Constants§

DESCRIPTOR
The connector’s static descriptor (RFC 0013): its capabilities, answered from config alone.