Skip to main content

Crate faucet_source_pubsub

Crate faucet_source_pubsub 

Source
Expand description

§faucet-source-pubsub

Google Cloud Pub/Sub source connector for faucet-stream: streaming pull from a subscription, per-message record assembly with a configurable value_format (json / string / bytes) and an attribute map surfaced under a configurable key, and the standard idle_termination_secs / max_messages termination knobs (at least one is required).

Delivery is at-least-once. Messages are acked only at durable page boundaries — a page’s messages are acked once the pipeline has written that page to the sink and persisted its bookmark, so a crash between the sink write and the ack redelivers those messages on the next run. Pair with an upsert sink keyed on message_id when replays must converge. Exactly-once delivery is out of scope (Pub/Sub provides no compatible primitive).

Structs§

PubsubBookmark
The persisted (informational) bookmark value.
PubsubConnection
Connection settings shared by the Pub/Sub source and sink. Flattened into each connector config via #[serde(flatten)], so project_id / endpoint / emulator_host / credentials appear at the config top level.
PubsubSource
Google Cloud Pub/Sub source. See the crate README for semantics.
PubsubSourceConfig
Configuration for PubsubSource.

Enums§

PubsubCredentials
How to authenticate with Google Cloud Pub/Sub.
ValueFormat
How each message’s data payload is decoded into the emitted data field.

Constants§

DEFAULT_ATTRIBUTES_KEY
The default JSON key the per-message attribute map is surfaced under.