pub struct SnowflakeSourceConfig {
pub account: String,
pub warehouse: String,
pub database: String,
pub schema: String,
pub role: Option<String>,
pub auth: AuthSpec<SnowflakeAuth>,
pub query: String,
pub params: Vec<Value>,
pub statement_timeout: Duration,
pub poll_timeout: Duration,
pub batch_size: usize,
}Expand description
Configuration for the Snowflake query source.
Fields§
§account: StringSnowflake account identifier (e.g. "xy12345.us-east-1").
warehouse: StringWarehouse to use for the session.
database: StringDatabase name.
schema: StringSchema name.
role: Option<String>Optional role to assume for the session.
auth: AuthSpec<SnowflakeAuth>Authentication: either inline ({ type, config }) or a { ref: <name> }
pointer to a shared provider in the CLI’s top-level auth: catalog.
A shared provider must yield a Bearer or Token credential, which
maps onto SnowflakeAuth::OAuth; key-pair JWT is always inline.
query: StringSQL statement to execute. May contain ${field.path} placeholders that
are resolved against the parent-record context at runtime; resolved
values are sent as positional bind parameters appended after
params.
params: Vec<Value>Positional bind parameters for the query, applied in order before any
context-derived values. Snowflake’s SQL REST API uses 1-based positional
binds in the JSON request body (see the bindings field in the
SQL API docs).
statement_timeout: DurationPer-statement server-side timeout. Defaults to 60 seconds. Passed
through as the timeout field on the POST /api/v2/statements request
body. The HTTP-level timeout for each individual request is configured
separately by the source via the underlying reqwest client defaults.
poll_timeout: DurationMaximum wall-clock time the source will spend polling an asynchronous
statement (one for which the initial POST /api/v2/statements
returns HTTP 202) before giving up with
FaucetError::Source.
Without this cap a statement that never finishes would loop forever.
Defaults to 300 seconds. Set to 0 to disable the cap and poll
indefinitely.
batch_size: usizeRecords per emitted StreamPage.
Snowflake’s SQL REST API splits large result sets into partitions (one
chunk of rows per HTTP response). The source re-frames partitions into
batch_size-sized pages: rows accumulate in a buffer and are yielded as
soon as the buffer reaches batch_size. Defaults to
DEFAULT_BATCH_SIZE.
batch_size = 0 is the “no batching” sentinel: the entire result
set is buffered and emitted in a single page. Useful for small lookup
tables, or for sinks (e.g. SQL COPY, BigQuery load jobs) that prefer
one large request to many small ones.
Implementations§
Source§impl SnowflakeSourceConfig
impl SnowflakeSourceConfig
Sourcepub fn new(
account: impl Into<String>,
warehouse: impl Into<String>,
database: impl Into<String>,
schema: impl Into<String>,
auth: SnowflakeAuth,
query: impl Into<String>,
) -> Self
pub fn new( account: impl Into<String>, warehouse: impl Into<String>, database: impl Into<String>, schema: impl Into<String>, auth: SnowflakeAuth, query: impl Into<String>, ) -> Self
Create a new config with required fields and sensible defaults.
Sourcepub fn with_params(self, params: Vec<Value>) -> Self
pub fn with_params(self, params: Vec<Value>) -> Self
Set positional bind parameters for the query.
Sourcepub fn with_statement_timeout(self, timeout: Duration) -> Self
pub fn with_statement_timeout(self, timeout: Duration) -> Self
Set the per-statement server-side timeout.
Sourcepub fn with_poll_timeout(self, timeout: Duration) -> Self
pub fn with_poll_timeout(self, timeout: Duration) -> Self
Set the maximum wall-clock time spent polling an asynchronous
statement before giving up. Pass Duration::ZERO to poll forever.
Sourcepub fn with_batch_size(self, batch_size: usize) -> Self
pub fn with_batch_size(self, batch_size: usize) -> Self
Set the records-per-page hint for Source::stream_pages.
Pass 0 to opt out of batching — the entire result set is emitted in
a single page.
Trait Implementations§
Source§impl Clone for SnowflakeSourceConfig
impl Clone for SnowflakeSourceConfig
Source§fn clone(&self) -> SnowflakeSourceConfig
fn clone(&self) -> SnowflakeSourceConfig
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for SnowflakeSourceConfig
impl Debug for SnowflakeSourceConfig
Source§impl<'de> Deserialize<'de> for SnowflakeSourceConfig
impl<'de> Deserialize<'de> for SnowflakeSourceConfig
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Source§impl JsonSchema for SnowflakeSourceConfig
impl JsonSchema for SnowflakeSourceConfig
Source§fn schema_id() -> Cow<'static, str>
fn schema_id() -> Cow<'static, str>
Source§fn json_schema(generator: &mut SchemaGenerator) -> Schema
fn json_schema(generator: &mut SchemaGenerator) -> Schema
Source§fn inline_schema() -> bool
fn inline_schema() -> bool
$ref keyword. Read more