Module ingest 

Ingest inline data (JSON strings, CSV strings) and CSV files into Hyper tables.

JSON is inserted row-by-row via SQL INSERT statements. This is simple and correct but not the fastest path — for bulk data, prefer file-based ingest where Hyper’s native COPY FROM is used.

CSV ingest always uses COPY FROM: inline CSV is spilled to a temp file first, while file-based CSV is read directly by hyperd.

Atomicity

Every ingest function wraps its INSERT / COPY work inside a single transaction via Engine::execute_in_transaction. If any row fails to insert, all prior inserts from the same call are rolled back, so a failed ingest leaves zero additional rows behind.

Note that Hyper auto-commits DDL (DROP TABLE, CREATE TABLE) regardless of the surrounding transaction. In replace mode, this means the original table is already gone by the time inserts start — a mid-ingest failure leaves the new table empty, not the original intact. In append mode, no DDL runs (assuming the table already exists), so rollback is fully atomic.

Structs

IngestOptions

Controls how data is loaded into a target table.

IngestResult

Returned by every ingest function with the row count, resolved schema, and performance telemetry.

Enums

InferredFileFormat

High-level file-format categories the ingest and inspect layers dispatch on. Text-based formats (JSON, CSV) are distinguished by peeking at the first non-whitespace byte when the extension is unfamiliar, so log-like files with .log/.txt/no extension still reach the right decoder without the caller having to rename them.

Functions

detect_file_format

Decide how to dispatch an ingest / inspect call for path.

extract_json_path

Navigate a dot-separated path into a JSON value, transparently parsing stringified JSON at each step.

ingest_csv

Ingest inline CSV text into a Hyper table.

ingest_csv_file

Ingest a CSV file from disk into a Hyper table.

ingest_csv_file_async

Async twin of ingest_csv_file. Runs schema inference on the blocking pool, then issues CREATE TABLE + COPY FROM on the given async connection inside a single transaction.

ingest_json

Ingest an inline JSON array of objects into a Hyper table.

ingest_json_async

Async twin of ingest_json: insert a JSON array of objects on the given async connection. See ingest_json for the JSON-shape contract (expects a top-level array; call normalize_json_or_jsonl first if you have JSONL).

ingest_json_file

Ingest a JSON or JSON-Lines file from disk into a Hyper table.

ingest_json_file_async

Async twin of ingest_json_file.

merge_via_temp_table

Implement mode = "merge" for any format by reusing that format’s replace-mode load to populate a temp table, then upserting the target by merge_key columns.

normalize_json_or_jsonl

Normalize a raw JSON / JSONL string to the “JSON array of objects” representation ingest_json expects. Returns the original string when it already parses as a top-level array; otherwise treats the input as JSONL and re-serializes into a JSON array.

qualified_table

Build a SQL table identifier from IngestOptions. When target_db is set, returns "db"."public"."table"; otherwise "table" (unqualified).