Skip to main content

Module target

Module target 

Source
Expand description

Target-type resolver (ADR-0014 L4, roadmap §16).

Given a column’s canonical RivetType and a runtime-chosen ExportTarget, resolve what the column becomes in a downstream warehouse: the native target type (full fidelity), the autoload type a generic Parquet reader infers without a declared schema, a safety TargetStatus, a note, and an optional materialization (cast_sql / load-schema hint).

Design (locked in the type-support architecture review):

  • Dispatch on RivetType, never the physical Arrow type. The previous bq_compat matched on arrow::DataType and so was blind to json / uuid / enum (all Utf8 / FixedSizeBinary by then) and hard-failed UUID. The resolver keys off the semantic type; Arrow is consulted only for decimal precision (and RivetType::Decimal already carries p,s).
  • Total & infallible. Every (RivetType, ExportTarget) pair yields a populated TargetColumnSpec; an unmappable column is a status: Fail row, never an Err. This keeps the type-report table and --json shape stable.
  • autoload_type tells the truth. It encodes the empirically verified behavior of each target’s Parquet autoloader — notably that BigQuery autoload degrades Parquet JsonType/UUIDType to BYTES, isAdjustedToUTC=false timestamps to TIMESTAMP (not DATETIME), and 3-level lists to REPEATED RECORD{item}. DuckDB honors all of them.

BigQuery numeric limits (as of 2025): NUMERIC — precision 1–29, scale 0–9 BIGNUMERIC — precision 1–76, scale 0–38

Structs§

TargetColumnSpec
One column’s per-target materialization spec (ADR-0014 L4). Uniform across targets so the type-report table and --json stay stable; an unmappable column is a status: Fail row, not an error.
TargetInput
The borrowed subset a resolver is allowed to read. Built from a TypeMapping via From. Dispatch is on rivet_type; arrow_type is retained for callers that want it but the resolver reads precision from RivetType::Decimal directly.

Enums§

ExportTarget
A supported downstream warehouse target. Closed, in-tree, contract-tested set; chosen at runtime from --target X, one per run.
TargetStatus
Status of a column’s resolution against a specific target.