Skip to main content

TargetLoader

rivet::load

Trait TargetLoader 

Source 
pub trait TargetLoader {
    // Required methods
    fn fqtn(&self, table: &str) -> String;
    fn materialize(
        &self,
        table: &str,
        specs: &[TargetColumnSpec],
        uris: &[String],
    ) -> Result<u64>;
    fn append_changelog(
        &self,
        table: &str,
        specs: &[TargetColumnSpec],
        uris: &[String],
        pk: &[String],
    ) -> Result<u64>;
    fn warehouse(&self) -> Warehouse;
    fn create_view(&self, table: &str, view_sql: &str) -> Result<()>;
}
Expand description

A warehouse adapter — the small, warehouse-specific seam the driver drives. Dialect + CLI (bq / snow), the external stage, BigQuery’s 4,000-partition batch split, and PARSE_JSON all live behind these primitives.

Idempotent under retry: Rivet is at-least-once at the file layer, so the same Parquet object may be presented more than once; materialize overwrites.

Required Methods§

Source

fn fqtn(&self, table: &str) -> String

Fully-qualify table for this warehouse (project.dataset.t / db.schema.t).

Source

fn materialize( &self, table: &str, specs: &[TargetColumnSpec], uris: &[String], ) -> Result<u64>

Overwrite table with the Parquet at uris, materializing the native column types in specs. Returns the rows the load landed.

Source

fn append_changelog( &self, table: &str, specs: &[TargetColumnSpec], uris: &[String], pk: &[String], ) -> Result<u64>

Append the CDC change Parquet into <table>__changes (created if absent), prepending the __op / __pos / __seq meta columns to specs. Returns the rows this call appended.

Source

fn warehouse(&self) -> Warehouse

The warehouse this adapter targets — lets the shared driver build the current-state view SQL (dialect keyword + identifier quoting) in ONE place per mode instead of once per adapter.

Source

fn create_view(&self, table: &str, view_sql: &str) -> Result<()>

CREATE OR REPLACE the current-state view <table> from pre-built view_sql (the driver builds it via cdc::dedup_view_sql for CDC or cdc::inc_dedup_view_sql for incremental). The adapter only executes it its way (e.g. Snowflake prefixes a QUERY_TAG).

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§