Module cdc 

CDC current-state dedup view.

rivet CDC appends a change log to <table>__changes (free LOAD DATA / billed COPY); a view collapses it to current state. The collapse is one ROW_NUMBER window that keeps the latest change per PK:

ROW_NUMBER() OVER (PARTITION BY <pk> ORDER BY <total change order> DESC) = 1

A deleted row is kept as a tombstone, not dropped: the winning change’s __op becomes a boolean __is_deleted column, so the row survives with its last-known values and an auditable delete flag — a delete is never a silent disappearance. Live current state is WHERE NOT __is_deleted.

The total change order is (__pos, __seq):

• __pos is the commit position — it orders changes across transactions, but every change in one transaction shares it (verified live on all three engines: 8000 updates of one PK in one transaction → a single __pos).
• __seq (OSS TxnSeq) is the intra-transaction ordinal — it breaks that tie. Without it the dedup picked an arbitrary row (live: counter = 1 for a row whose committed value was 8000).

__pos is a JSON string whose shape is per source engine, so its parse (the part before __seq) is engine-specific — see SourceEngine. The parse functions are also warehouse-specific (Warehouse): BigQuery reads JSON with JSON_VALUE, Snowflake with PARSE_JSON(...):path.

Enums

SourceEngine

The source engine a change log came from — selects how __pos is parsed into a sortable key.

Warehouse

The warehouse the view is defined in — selects the JSON-parse dialect and the SELECT * EXCEPT/EXCLUDE keyword.

Constants

DELETE_FLAG_COLUMN

The soft-delete flag column the view exposes: true when the latest change for a PK was a delete. In rivet’s reserved __ namespace so it can never collide with a source column (a plain is_deleted might).

Functions

dedup_view_sql

Build the current-state dedup view over a <table>__changes log for warehouse. pk is the change log’s primary key column(s); engine selects the __pos parse. The view is free to define; reading it scans __changes (billed), kept cheap by clustering the log on pk.

inc_dedup_view_sql

Build the current-state dedup view for an incremental load’s change log. Unlike CDC (dedup_view_sql), an incremental delta has no __op/__pos/ __seq (the change log reuses the CDC append so those columns exist but are NULL): current state is simply the row with the greatest cursor_column per PK. Incremental can’t observe deletes, so DELETE_FLAG_COLUMN is a constant FALSE — the view SHAPE matches CDC so downstream reads WHERE NOT __is_deleted uniformly across both modes.

meta_column_specs

The three CDC meta columns rivet’s change log carries, typed for warehouse. rivet CDC writes __op (Utf8), __pos (Utf8), __seq (Int64) ahead of the after-image columns (OSS cdc::sink); rivet check reports only the data columns, so the loader must prepend these to build the <table>__changes schema.