Skip to main content

run_load_incremental

Function run_load_incremental 

Source
pub fn run_load_incremental(
    loader: &dyn TargetLoader,
    table: &str,
    specs: &[TargetColumnSpec],
    uris: &[String],
    pk: &[String],
    cursor_column: &str,
    expected_delta: Option<u64>,
    cleanup: Option<(&GcsStore, &str)>,
) -> Result<CdcLoadReport>
Expand description

Load an INCREMENTAL export’s delta: APPEND the parquet into <table>__changes (reusing the CDC changelog append — the delta’s rows land with NULL __op/ __pos/__seq, which the view drops) and (re)build a current-state view deduped to the latest row per PK by cursor_column. The manifests’ summed row_count gates the appended delta, and cleanup runs (only) after the gate — safe because the ledger, not the file prefix, records what’s loaded.