Module schema 

Schema inference and type mapping.

Three-Tier Inference

The tier is selected automatically based on the data source:

Tier	Source	Strategy
Exact	Arrow IPC, Parquet	Types read from file metadata — zero guessing.
Structural	JSON	Full scan of all objects. Per-column type widening (see below).
Heuristic	CSV	Header row for names, first 1 000 rows sampled for types. Ambiguous → TEXT.

All tiers can be bypassed with an explicit schema override from the caller.

Type Widening Rules (Structural / Heuristic Tiers)

When a JSON or CSV column contains mixed types across rows, the widening chain determines the final type:

Null < Bool < Int < BigInt < Double < Date < Timestamp < Text

Specific rules (implemented in resolve_type):

• All null → TEXT (safe catch-all).
• Uniform non-null → that type, unchanged.
• Mixed numeric (Int / BigInt / Double) → widest numeric type seen.
• Any other mix (e.g. Bool + Int, Date + Text) → TEXT.

Arrow / Parquet Type Mapping

The exact-tier mapping from Arrow types to Hyper SQL types is handled by crate::ingest_arrow::arrow_type_to_hyper. Key mappings:

Arrow	Hyper
Int16/Int32/Int64	SMALLINT/INT/BIGINT
UInt16/UInt32/UInt64	INT/BIGINT/BIGINT (promoted to signed)
Float32/Float64	DOUBLE PRECISION
Utf8/LargeUtf8	TEXT
Date32/Date64	DATE
Timestamp(_, None/Some)	TIMESTAMP / TIMESTAMPTZ
Decimal128/Decimal256	NUMERIC

Structs

ColumnSchema

A column’s name, Hyper SQL type (as a string like "INT" or "DOUBLE PRECISION"), and nullability. This is the internal schema representation shared across all ingest and table-creation paths.

Functions

apply_schema_override

Overlay a user-provided override ({"column_name": "TYPE", ...}) on top of an already-inferred column list, preserving the inferred column order and replacing only the types listed in the override.

build_table_def

Build a hyperdb-api TableDefinition from a list of ColumnSchema.

infer_csv_schema

Infer a ColumnSchema for each CSV column (Tier 3).

infer_json_schema

Infer a ColumnSchema for each key in a JSON array of objects (Tier 2).

map_hyper_type

Map a user-facing type name string (e.g. "INT", "NUMERIC(12,2)") to a hyperdb_api::SqlType. Accepts PostgreSQL aliases (INT4, FLOAT8, VARCHAR) so schema overrides from diverse sources work without normalization.

normalize_schema_param

Normalize a raw MCP schema parameter value into the column-name → type map expected by apply_schema_override and parse_schema_override.

parse_schema_override

Parse a user-provided schema override ({"column_name": "TYPE", ...}) into a Vec<ColumnSchema>. Validates each type name against map_hyper_type and rejects unknown types early. All override columns are marked nullable.

widen_csv_numeric_columns

Second-pass streaming widen: re-read the given CSV source and, for columns the first-pass inference classified as INT, BIGINT, or DOUBLE PRECISION, promote the type if a value outside its current range appears anywhere in the file (not just the first 1 000 rows).