Skip to main content

Module create_table

Module create_table 

Source
Expand description

Shared auto-create-table planning for table-based sinks (#580).

A first-ever sync cannot assume the destination table exists. The BigQuery sink solved this in #578 by inferring a schema from the first written page and creating the table; every other table destination has the same problem, and before this module each one answered it differently (or not at all) — different field names, different defaults, some with no create path at all.

This module holds the dialect-neutral half: turn a page of records into an ordered, deduplicated column plan. Each sink supplies only the mapping from SqlBaseType to its own type keyword — the same mapping its evolve_schema already needs, so the two agree by construction rather than by inspection.

§Why column order is sorted, not source order

Source order would read better in a SELECT *, but it is not recoverable: serde_json::Map is a BTreeMap or an IndexMap depending on whether the preserve_order feature is unified into the build, so “the order the record listed its keys” is a build-time property. A created table’s column order would then differ between a -p crate build and an --all-features one — the kind of difference that only shows up as a diffed schema in production. Sorted is worse to read and always the same.

Structs§

PlannedColumn
One column of a planned CREATE TABLE.

Functions§

missing_target_error
The uniform error a sink raises when its target is missing and create_table: false (#580).
plan_columns
Plan the columns for a table created from page.
plan_keyed_columns
Plan the columns for a table whose writes dedup on key (#676).
render_column_defs
Render a column list where the type keyword may depend on the column, not just its base type — for dialects that cannot index an unbounded type and so need a bounded one for key columns (MySQL TEXT, SQL Server NVARCHAR(MAX)).
render_columns
Render a CREATE TABLE column list, given a dialect’s type mapper and identifier quoter.
render_primary_key
PRIMARY KEY (<key…>) for a created table, or None for an empty key.