Light Snowflake Connector
Minimal wrapper around Snowflake's public REST API.
- Easy to use
- Intended for small, tactical queries and point lookups,
- Particularly for dashboards and other interactive use cases
- Keep in mind that Snowflake latency is usually >100ms so design carefully
- Designed not to break for large result sets, but not optimized for them either
Usage
Add following line to Cargo.toml:
# Cargo.toml
= "0.1.0"
use ;
use RS256KeyPair;
async
Features & Limitations
Authentication:
- Key Pair Authentication
- OAuth, SSO: This is possible but not implemented yet
- Username/Password: Not available in Snowflake's REST API 2.0
Querying:
- Prepared Statements with
qmark
"?" Bindings- No other bindings are supported
- Rust
async
support (but synchronous from Snowflake's point of view) - Snowflake "async" support (for super long running queries)
- GET and PUT: not supported by Snowflake's REST API 2.0
- Arrow support: we're trying to keep the dependency tree small
- Streaming support, and multiple batches
Types:
- String, str
- i128
- f64
- bool
- Date, Time, Timestamp_Ntz (NaiveDateTime), Timestamp_Ltz (DateTime; not well testes, not sure about the use cases)
- Timestamp_Tz (DateTime)
- Decimal (dec and rust_decimal have different semantics and precision)
Implicit Type Conversions
Snowflake's NUMBER type is 128 bit (38 decimal digits) but supports a scale as well. There's no native Rust type that can achieve both of these so we opted for the more convenient (and probably common) use cases:
- Integers are converted to i128, which is lossless
- Floats are converted to f64, which is lossy
- Which to do is determined on a cell by cell basis, so you can have a column with mixed types
This particular workaround could be improved by using a Decimal type, but there are some issues with the available libraries:
- rust_decimal is a pure Rust implementation, but it doesn't support 128 bit numbers
- dec supports 128 bit numbers, but somehow 4 digits less decimal precision. Also, it's a wrapper around a C library, so it could cause issues downstream for WASM users (e.g. FaaS)
- arrow (and FWIW, arrow2) supports 128 bit numbers, but it's a huge dependency and we'd have to pivot to columnar data structures and a different API.
Multiple Batches
This library supports multiple batches, which is useful for streaming large result sets. But the results are transferred as JSON, so if high throughput is a concern, you should consider one of the Arrow based libraries instead, like snowflake-api.
Relationship to other Snowflake Connectors
This is a fork of the snowflake-connector library, and differs in a few ways:
- It returns Cells rather than deserializing the data into custom structs
- It does not support Decimal types (because it is hard to do correctly)
It differs from snowflake-api in that:
- It uses only the documented v2 API, which (presently) does not support GET or PUT
- It doesn't use any deprecated or undocumented APIs at all
- It doesn't require Arrow (but it also doesn't get the performance benefits of Arrow)
It differs from most other languages' Snowflake connectors in that:
- It doesn't use Arrow
- It doesn't use the deprecated v1 API
- It doesn't use undocumented APIs
- It doesn't support GET or PUT
- It doesn't support Async Queries
- It's not affiliated with or supported by Snowflake.
- It's not an official product of any company, doesn't have any guarantees, warranties, or support.