Module projector 

Projector — the pluggable ingest seam, analogous to DuckDB’s table functions / replacement scans.

DuckDB queries CSV/Parquet/JSON uniformly because every reader lowers to a common DataChunk that the SQL engine consumes. Here every file format lowers to a common Situation stream (tokens + display cells) that the IKL/bitmap engine consumes — so adding a format never touches the query layer. Projection is push-based: a reader emits one Situation per source record into a sink.

The core owns the tabular/columnar/text projectors (CSV, JSONL text situations; Parquet/plain-text next). Heavy document formats (PDF/DOCX/PPTX/HTML) reuse the existing producer pipeline, which emits the same Situation stream over a sidecar bridge — no re-implementation of document parsing here.

Structs

Situation

One projected record: the tokens it asserts (for set-algebra), the cells to show for it, and any numeric fields it carries (for (num <field> <op> <value>) range predicates — e.g. a CSV numeric column or a canonicalised quantity span like qty-length = 70 m).

Enums

CorpusKind

Traits

Projector

A push-based reader over one source. project drives the sink once per source record.

Functions

slug

slugify a value into a token-safe form: trim, lowercase, collapse whitespace// to -.