Skip to main content

SnowflakeSourceConfig

Struct SnowflakeSourceConfig 

Source
pub struct SnowflakeSourceConfig {
    pub account: String,
    pub warehouse: String,
    pub database: String,
    pub schema: String,
    pub role: Option<String>,
    pub auth: AuthSpec<SnowflakeAuth>,
    pub query: String,
    pub params: Vec<Value>,
    pub statement_timeout: Duration,
    pub poll_timeout: Duration,
    pub batch_size: usize,
}
Expand description

Configuration for the Snowflake query source.

Fields§

§account: String

Snowflake account identifier (e.g. "xy12345.us-east-1").

§warehouse: String

Warehouse to use for the session.

§database: String

Database name.

§schema: String

Schema name.

§role: Option<String>

Optional role to assume for the session.

§auth: AuthSpec<SnowflakeAuth>

Authentication: either inline ({ type, config }) or a { ref: <name> } pointer to a shared provider in the CLI’s top-level auth: catalog. A shared provider must yield a Bearer or Token credential, which maps onto SnowflakeAuth::OAuth; key-pair JWT is always inline.

§query: String

SQL statement to execute. May contain ${field.path} placeholders that are resolved against the parent-record context at runtime; resolved values are sent as positional bind parameters appended after params.

§params: Vec<Value>

Positional bind parameters for the query, applied in order before any context-derived values. Snowflake’s SQL REST API uses 1-based positional binds in the JSON request body (see the bindings field in the SQL API docs).

§statement_timeout: Duration

Per-statement server-side timeout. Defaults to 60 seconds. Passed through as the timeout field on the POST /api/v2/statements request body. The HTTP-level timeout for each individual request is configured separately by the source via the underlying reqwest client defaults.

§poll_timeout: Duration

Maximum wall-clock time the source will spend polling an asynchronous statement (one for which the initial POST /api/v2/statements returns HTTP 202) before giving up with FaucetError::Source. Without this cap a statement that never finishes would loop forever. Defaults to 300 seconds. Set to 0 to disable the cap and poll indefinitely.

§batch_size: usize

Records per emitted StreamPage.

Snowflake’s SQL REST API splits large result sets into partitions (one chunk of rows per HTTP response). The source re-frames partitions into batch_size-sized pages: rows accumulate in a buffer and are yielded as soon as the buffer reaches batch_size. Defaults to DEFAULT_BATCH_SIZE.

batch_size = 0 is the “no batching” sentinel: the entire result set is buffered and emitted in a single page. Useful for small lookup tables, or for sinks (e.g. SQL COPY, BigQuery load jobs) that prefer one large request to many small ones.

Implementations§

Source§

impl SnowflakeSourceConfig

Source

pub fn new( account: impl Into<String>, warehouse: impl Into<String>, database: impl Into<String>, schema: impl Into<String>, auth: SnowflakeAuth, query: impl Into<String>, ) -> Self

Create a new config with required fields and sensible defaults.

Source

pub fn with_role(self, role: impl Into<String>) -> Self

Set the session role.

Source

pub fn with_params(self, params: Vec<Value>) -> Self

Set positional bind parameters for the query.

Source

pub fn with_statement_timeout(self, timeout: Duration) -> Self

Set the per-statement server-side timeout.

Source

pub fn with_poll_timeout(self, timeout: Duration) -> Self

Set the maximum wall-clock time spent polling an asynchronous statement before giving up. Pass Duration::ZERO to poll forever.

Source

pub fn with_batch_size(self, batch_size: usize) -> Self

Set the records-per-page hint for Source::stream_pages.

Pass 0 to opt out of batching — the entire result set is emitted in a single page.

Trait Implementations§

Source§

impl Clone for SnowflakeSourceConfig

Source§

fn clone(&self) -> SnowflakeSourceConfig

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for SnowflakeSourceConfig

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl<'de> Deserialize<'de> for SnowflakeSourceConfig

Source§

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>
where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer. Read more
Source§

impl JsonSchema for SnowflakeSourceConfig

Source§

fn schema_name() -> Cow<'static, str>

The name of the generated JSON Schema. Read more
Source§

fn schema_id() -> Cow<'static, str>

Returns a string that uniquely identifies the schema produced by this type. Read more
Source§

fn json_schema(generator: &mut SchemaGenerator) -> Schema

Generates a JSON Schema for this type. Read more
Source§

fn inline_schema() -> bool

Whether JSON Schemas generated for this type should be included directly in parent schemas, rather than being re-used where possible using the $ref keyword. Read more
Source§

impl Serialize for SnowflakeSourceConfig

Source§

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>
where __S: Serializer,

Serialize this value into the given Serde serializer. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> DeserializeOwned for T
where T: for<'de> Deserialize<'de>,

Source§

impl<T> DynClone for T
where T: Clone,

Source§

fn __clone_box(&self, _: Private) -> *mut ()

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> PolicyExt for T
where T: ?Sized,

Source§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow only if self and other return Action::Follow. Read more
Source§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow if either self or other returns Action::Follow. Read more
Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V

Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more