Skip to main content

Crate faucet_source_databricks

Crate faucet_source_databricks 

Source
Expand description

§faucet-source-databricks

Databricks SQL query source for the faucet-stream ecosystem. Runs a SQL statement against a Databricks SQL Warehouse via the Statement Execution API (plain REST — no JDBC/ODBC driver) and streams the result rows as typed JSON objects.

This is the query-results read path for Databricks (joins / aggregates / filtered extracts). For full-table lakehouse scans and the write path, use the Delta Lake connectors (faucet-source-delta / faucet-sink-delta).

  • Async statement lifecycle: submit → poll until terminal → stream result chunks (INLINE + JSON_ARRAY), following next_chunk_internal_link.
  • Type-aware decode from the response manifest column schema.
  • Incremental replication via a bookmark column + a ${bookmark} token (server-side named param) plus a client-side filter backstop.
  • Bearer auth (PAT / OAuth M2M), inline or via the shared auth: catalog.

Structs§

DatabricksParam
A named SQL parameter passed to the statement (:name markers in the SQL).
DatabricksSource
Databricks SQL query source.
DatabricksSourceConfig
Configuration for the Databricks SQL query source.

Enums§

DatabricksAuth
Authentication for the Databricks SQL Statement Execution API.
DatabricksReplication
How the source replicates rows across runs.