Struct SqliteSink 

pub struct SqliteSink { /* private fields */ }

Available on crate feature sink-sqlite only.

A sink that writes JSON records to a SQLite table.

Implementations

impl SqliteSink

pub async fn new(config: SqliteSinkConfig) -> Result<SqliteSink, FaucetError>

Create a new SQLite sink. Establishes a connection pool.

The pool opens each connection with journal_mode = WAL and a 5-second busy_timeout. WAL lets a writer and readers proceed concurrently instead of locking each other out, and the busy timeout makes a connection wait-and-retry for the write lock rather than failing immediately with SQLITE_BUSY under contention. create_if_missing preserves the previous behaviour of creating the database file on first open. WAL on a sqlite::memory: database is a harmless no-op.

Trait Implementations

impl Sink for SqliteSink

fn check<'life0, 'life1, 'async_trait>( &'life0 self, ctx: &'life1 CheckContext, ) -> Pin<Box<dyn Future<Output = Result<CheckReport, FaucetError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, SqliteSink: 'async_trait,

Preflight connectivity probe (faucet doctor).

Acquires a connection from the existing pool and runs SELECT 1. This is non-mutating and idempotent — it validates that the database file / connection opens without writing anything.

fn current_schema<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Option<Value>, FaucetError>> + Send + 'async_trait>>

where 'life0: 'async_trait, SqliteSink: 'async_trait,

Read the live destination schema via PRAGMA table_info, shaped as an infer_schema-compatible object ({"type":"object","properties":{…}}), or None when the target table does not exist yet (issue #194).

PRAGMA table_info returns one row per column with name, type (the declared affinity string), and notnull. The affinity string is mapped to a JSON-Schema base type via sqlite_affinity_to_json_schema, and notnull == 0 surfaces the column as nullable. The PRAGMA runs on a connection acquired from the pool (a standalone read — not inside an open transaction).

fn evolve_schema<'life0, 'life1, 'async_trait>( &'life0 self, evolution: &'life1 SchemaEvolution, ) -> Pin<Box<dyn Future<Output = Result<(), FaucetError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, SqliteSink: 'async_trait,

Apply an additive schema evolution to the destination table (issue #194).

• Additions — ALTER TABLE … ADD COLUMN. SQLite has no ADD COLUMN IF NOT EXISTS, so the current columns are read first and a column already present is silently skipped (idempotency by pre-check).
• Widenings — a no-op under SQLite’s dynamic typing: a column already accepts a value of any type, so there is nothing to ALTER. Logged once at debug.
• Nullability relaxations — a no-op: SQLite cannot drop a NOT NULL constraint in place (it requires a full table rebuild, which is out of scope here). Logged once at debug; the column is left as-is.

fn write_batch_partial<'life0, 'life1, 'async_trait>( &'life0 self, records: &'life1 [Value], ) -> Pin<Box<dyn Future<Output = Result<Vec<Result<(), FaucetError>>, FaucetError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, SqliteSink: 'async_trait,

Write a batch and report per-row outcomes.

In append mode this delegates to write_batch and maps a single success onto an all-Ok(()) vector (the trait default). In upsert/delete mode the good rows are applied (upserts + deletes), and only the rows whose key could not be extracted (missing / null key) are reported as Err so the pipeline routes them to the DLQ per-row instead of sending the whole page.

fn config_schema(&self) -> Value

Return a JSON Schema describing the configuration this sink accepts.

fn dataset_uri(&self) -> String

Logical dataset identity for lineage emission, following OpenLineage naming conventions (https://openlineage.io/docs/spec/naming).

fn supported_write_modes(&self) -> &'static [WriteMode]

Write modes this sink can apply. Default: append-only. Sinks that implement key-based merge override this to include WriteMode::Upsert / WriteMode::Delete. The CLI rejects a configured mode that is not in this set at config-load time.

fn dedups_by_key(&self) -> bool

Whether this sink instance is configured to dedup by key — i.e. write_mode: upsert (or delete) with a non-empty key, so re-applying a record with the same key converges instead of duplicating. Default: false.

fn supports_schema_evolution(&self) -> bool

Whether this sink can apply additive/widening DDL via evolve_schema. Default: false. The CLI rejects on_drift: evolve against a sink that returns false at config-load.

fn write_batch<'life0, 'life1, 'async_trait>( &'life0 self, records: &'life1 [Value], ) -> Pin<Box<dyn Future<Output = Result<usize, FaucetError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, SqliteSink: 'async_trait,

Write a batch of records to the destination.

fn supports_idempotent_writes(&self) -> bool

Whether this sink can durably commit a page’s rows and a commit token in a single atomic transaction. Default: false (at-least-once only).

fn last_committed_token<'life0, 'life1, 'async_trait>( &'life0 self, scope: &'life1 str, ) -> Pin<Box<dyn Future<Output = Result<Option<String>, FaucetError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, SqliteSink: 'async_trait,

The last token durably committed for scope, or None if this sink has never committed under that scope. Default: None.

fn write_batch_idempotent<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, records: &'life1 [Value], scope: &'life2 str, token: &'life3 str, ) -> Pin<Box<dyn Future<Output = Result<usize, FaucetError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait, SqliteSink: 'async_trait,

Write records AND durably record token for scope, atomically.

fn flush<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<(), FaucetError>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

Flush any buffered data to the destination.

fn sink_guarantee(&self) -> SinkGuarantee

The strongest delivery guarantee this sink can uphold — see SinkGuarantee.

fn connector_name(&self) -> &'static str

Stable identifier used as the connector label on metrics and the connector attribute on spans. See Source::connector_name.