Skip to main content

Crate firn

Crate firn 

Source
Expand description

§firn

Rust client for Snowflake’s internal HTTP API (the same endpoint the official drivers use). Forked from snowflake-api at v0.14.0 (andrusha/snowflake-rs).

[dependencies]
firn = "0.15"

Default features: cert-auth. Optional: browser-auth, polars.

§Quick start

use firn::{QueryData, SnowflakeApi};

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let api = SnowflakeApi::from_env()?;
    let r = api.exec("SELECT current_version()").await?;
    match r.data {
        QueryData::Arrow(batches) => { /* ... */ }
        QueryData::Json(v) => { /* ... */ }
        QueryData::Empty => {}
    }
    Ok(())
}

§Features

§Auth

§Queries

§Results

  • Arrow RecordBatch, with streaming and raw-IPC variants (streaming.rs)
  • JSON results when the session is configured for JSON
  • per-query QueryMetadata: query_id, total_rows, total_chunks, statement_type_id, executing warehouse/database/schema/role
  • cast_structured() rewrites MAP / OBJECT / ARRAY columns from JSON-in-Utf8 into native Arrow Map<Utf8, V> / List<E> (compound_types.rs)
  • GEOGRAPHY / GEOMETRY carried via FieldSchema::ext_type_name; VECTOR via vector_dimension + element type
  • StatementType enum, is_dql() predicate

§Cancellation

§Session

  • session-keep-alive heartbeat (with_keep_alive) (keep_alive.rs)
  • session-token renewal on 390112 mid-flight
  • parallel queries on a shared SnowflakeApi with a lock-free hot path (arc-swap) (parallel_queries.rs)

§Connection

  • retry middleware that rotates request_guid per attempt and writes retryCount / retryReason / clientStartTime on retried query-request calls
  • configurable connect and request timeouts
  • credentials and auth tokens wrapped in SecretString (Debug-redacted, zeroized on drop)
  • custom reqwest middleware injection (tracing/)

§PUT

  • AWS S3 storage backend
  • glob expansion (PUT 'file:///tmp/*.csv' @stage)
  • parallel upload of small files

§Integrations

  • polars DataFrame conversion (polars feature) (polars/)

§Why

Snowflake exposes two HTTP APIs: the public SQL REST API and the undocumented endpoint that the official drivers use. This crate targets the latter, since it supports Arrow output and PUT/GET.

The wire format and retry/cancel semantics follow gosnowflake, the Go driver, since it outputs Arrow by default and is the most legible reference implementation.

§License

Apache-2.0. Original work © Andrew Korzhuev (andrusha/snowflake-rs); fork modifications © Will Eaton. See LICENSE.

Modules§

connection

Structs§

AuthArgs
Bind
A typed bind value. Construct via the inherent constructors (Bind::text, Bind::fixed, etc.) or via Into from common primitive types. Pass a slice of these to the query builder.
CertificateArgs
FieldSchema
Based on the [ExecResponseRowType]. Carries the full type-info surface Snowflake returns for a column — length / byte_length / extension-type discriminator / VECTOR dimension / nested element sub-schema — so codegen tools that consume describe() output don’t have to fall back to the raw response.
JsonResult
Even if Arrow is specified as a return type non-select queries will return Json array of arrays: [[42, "answer"], [43, "non-answer"]].
PasswordArgs
QueryBuilder
Fluent builder returned by SnowflakeApi::query. Accumulates bind parameters and optional cancellation/request-id, then runs the query via execute() (deserialized Arrow) or execute_raw() (raw bytes).
QueryHandle
Returned by SnowflakeApi::submit_async / QueryBuilder::submit_async. Both ids are useful: request_id is what SnowflakeApi::cancel_query understands (same-process abort path), query_id is what SnowflakeApi::query_status / SnowflakeApi::fetch_results expect (cross-process / cross-worker dispatch).
QueryMetadata
Per-query metadata Snowflake returns alongside the result body. Carried on QueryResult, RawQueryResult, and the streaming entry points so callers can correlate a result with its queryId (for history / debugging / cancel_query), see which session context actually executed it, and short-circuit on row count without scanning batches.
QueryResult
Container for a successful query response: QueryMetadata + rows. Arrow is returned by default for SELECTs unless the session is configured otherwise.
RawQueryResult
Raw counterpart to QueryResult: Arrow IPC bytes or JSON, plus metadata. Convertible into QueryResult via RawQueryResult::deserialize_arrow.
RecordBatch
A two-dimensional batch of column-oriented data with a defined schema.
SnowflakeApi
Snowflake API, keeps connection pool and manages session for you
SnowflakeApiBuilder

Enums§

ArrowError
Many different operations in the arrow crate return this error type.
AuthType
QueryData
QueryStatus
Mirrors gosnowflake’s monitoring.go query-status constants. Other is the catch-all so a new server-side status doesn’t immediately break callers; treat unknown values as non-terminal until proven otherwise.
RawQueryData
SnowflakeApiError
SnowflakeType
StatementType
Decoded statement_type_id, mirroring gosnowflake’s named constants (connection.go defines exactly these four; everything else is Other). gosnowflake’s grouping is intentionally coarse: INSERT / UPDATE / DELETE / MERGE all map to StatementType::Dml by way of being inside the [0x3000, 0x3500] range. If you need finer granularity, compare against the raw QueryMetadata::statement_type_id integer directly.

Functions§

cast_structured_batch
Rewrite Utf8 columns whose Arrow field metadata carries logicalType: "OBJECT" or "ARRAY" into proper Arrow Map<Utf8, V> / List<E>. V and E are inferred per-batch (Boolean / Int64 / Float64 / Utf8).
cast_structured_batch_with_schema
Like cast_structured_batch, but column_schema (the Snowflake rowtype carried on crate::QueryMetadata::column_schema) supplies ext_type_name info that isn’t in the Arrow field metadata yet. Use this on the streaming path so GEOGRAPHY / GEOMETRY columns stay as raw Utf8 GeoJSON instead of being shredded into Map<Utf8, Utf8>.
cast_structured_stream
Wrap a RecordBatchStream so each emitted batch is run through cast_structured_batch_with_schema with the supplied column_schema (typically meta.column_schema from the same execute_stream call). GEOGRAPHY / GEOMETRY columns stay raw, MAP / OBJECT / ARRAY columns become typed Arrow Map / List.

Type Aliases§

ArrowChunkStream
Stream of Arrow IPC blobs, one per Snowflake chunk, in original response order. Suitable for forwarding through HTTP/SSE without re-encoding.
RecordBatchStream
Stream of decoded Arrow RecordBatches. A single Snowflake chunk may contain multiple batches; they are flattened into the stream in order.
SecretString
Secret string type.